Chapter 4. Planning for the JAMstack

This section covers the JAMstack end-to-end setup and highlights the choices you’ll make along the way to create the appropriate setup for your team. These decisions include the following:

Let’s look at each one in turn.

Setting Up the Project

With no server or database to configure, a JAMstack project is largely a simple folder structure full of text files. The contents of your JAMstack project will usually include the following:

  • A source folder to store your content files

  • A folder for layouts and templates

  • A configuration file containing the settings for the build process

  • A static site generator, usually added as a dependency

  • A destination folder to save the final output

You can manage your entire project locally, directly on your computer, but it’s always best to begin by setting up a hosted Git repository. Even if you are the only site author, starting a repository will give you an online backup, store the history of the project, and eventually allow others to contribute. And in a moment, you’ll also see how a hosted repository is key to triggering new builds whenever a change is pushed.

Rarely would you set up your entire project folder from scratch because most site generators have sample projects that you can download from GitHub as a starting point. But before we talk about site generators, let’s cover the various ways to manage content on the JAMstack.

Strategies for Managing Content

Every website begins with content. On Content Management System (CMS) systems like Drupal or Wordpress, sites manage content in a database, and editors use an admin interface to write and save the content to the database.

On the JAMstack, there are actually a few content approaches.

Text Files

For many JAMstack sites, content couldn’t be more straightforward: it’s simply a folder full of files that lives directly within the project with the templates, plug-ins, and other assets. Creating a new file in the content directory will create a new page after the site is published, and changes are made by editing the content files in a text editor and then committing the changes to the Git repository. The name of each file and the folder it is in will determine its URL path after the site is generated.

What does one of these content files look like? It can be straight HTML—though usually paired down to only the unique parts of the page. Common elements like headers and footers then are added by the site generator during the build step.

Authoring site content in pure HTML can still be tedious, however. This is why Markdown, shown in Figure 4-1, a much-simplified syntax, has become the most popular option for writing content. If you’ve used Slack and surrounded a word with asterisks to *bold* it, you’ve used a modified form of markdown.

Markdown and the resulting HTML output
Figure 4-1. Markdown and the resulting HTML output

Most common CMSs have additional settings for each page; for example, which layout to use or which category and tags the page should be assigned.

To store details such as these, static site generators have standardized on a top section of each file called the metadata. Often written in YAML, it’s a simple key/value collection of settings and other data that apply to the page, as demonstrated in Figure 4-2.

A sample markdown file with metadata
Figure 4-2. A sample markdown file with metadata

Git-Based CMS

It’s likely that not all your site contributors will be developers, and using Git and text files may be an unfamiliar experience. This is actually the reason why database-backed CMSs like Wordpress were originally created. However, you probably also have experienced how a complex CMS can create artificial separation between the content editors working in the database and the developers building the code and templates.

Because every web page ultimately ends up as a combination of content and markup, it would be nice if you could manage them both by the same process.

There’s now a new generation of CMS software does exactly this, managing content as files in Git rather than a database. Usually included in your project as a single-page application (just an HTML file with some JavaScript dependencies), a Git-based CMS allows users to log in to a familiar admin interface to edit content and preview and save changes.

Here’s the clever bit: a saved change in the CMS is really made as a Git contribution behind the scenes. That’s powerful in that developers and editors are working in the same Git-based workstream, modifying the same files using whichever method is most comfortable to them.

NetlifyCMS.org is an open source CMS that works in this manner. Forestry.io is another.

Headless CMS

This is a category of hosted services that provide a rich online interface for managing and editing content.

“Headless” refers to the idea that these services manage only the content, leaving it up to the developer to determine how and where the site is built and hosted. Larger projects use a headless CMS as a central content store behind their entire collection of sites and mobile apps.

All headless CMS services provide an API that is used to pull the most recent content during the build step (and also dynamically using JavaScript as the site runs).

For example, you might use a headless CMS for your recruiting team to manage job postings. The static site generator would then query the API during the build step, receiving a list of the job postings as JSON. It would then use templates to create a unique HTML page for each posting, plus, of course, an index page listing all the postings together.

Self-Hosted CMS

Running the JAMstack does not preclude you from using something more traditional like Wordpress or Drupal to manage content. In recent versions, both of these popular CMSs have added APIs to access the content they contain. So, much like the aforementioned headless CMS service, you can connect your content to your JAMstack site using the API.

In this way, you can continue to manage content in a familiar CMS but now prebuild the site to dramatically speed up performance by serving rendered pages from a CDN. And becauseWordpress and Drupal would no longer need to be public facing, you have many more options to keep them secure.

Choosing a Site Generator

With content managed using one or more of the just described techniques, it’s time to think of how it is transformed into production HTML.

In the beginning, site generators were almost synonymous with Jekyll, the first site generator to gain wider traction and popularity. Jekyll was created by Tom Preston-Werner, GitHub’s cofounder, using the Ruby programming language.

Today, site generators are available in almost every language, including Ruby, Python, PHP, JavaScript, and Go. The website staticgen.com is a community-curated directory of static site generators, with more than 30 languages represented. Most all site generators are both free and open source.

Perhaps the most difficult part of getting started with the JAMstack is picking the appropriate site generator from the ever-expanding list of options. The staticgen.com website shows the popularity of each generator, but beyond what others think, there are some real factors that you should consider that will flavor your experience with the JAMstack.

One: The Type of Website

You’ll find each generator is created by its authors with a particular type of website or application in mind. Some generators are designed for websites with complex taxonomy and thousands of pages, others specifically for blogs, others for documentation, others for progressive web apps (PWAs), and richly interactive apps for which most all content and navigation is loaded and handled via JavaScript. Our advice is to check out sample projects created with each static site generator to see if they match your own intended results.

Two: The Programming Language Used

It’s very possible to use a site generator without knowing the programming language used to create it. For example, even if you know nothing about the Go programming language, Hugo is a great choice because you can download and install it as a binary application with nothing else to set up and no programming knowledge required.

Jekyll also does not require any specific Ruby experience to make good use of it, but you first need to have a working Ruby installation to get it to run on your computer. Gatsby, another popular site generator that uses the React JavaScript framework, requires you to have a recent version of Node.js installed. You get the idea: a little experience with the tooling around the language can help.

It’s not a bad idea to consider a site generator written in a language your team finds familiar. As your site grows more complex, you might find the need for your own custom plug-ins and transformations. Familiarity with the generator’s language can help you modify and extend its functionality.

Three: The Templating Syntax

A majority of your JAMstack development will be spent creating and modifying HTML templates, CSS, and JavaScript. Take a close look at the templating languages your chosen static site generator supports. All templating languages augment traditional HTML markup with new functionality like reusable code blocks, if statements, and loops. However, the exact syntax used varies and is largely a matter of personal preference. Some developers prefer the minimalist style of a language like Jade or Pug—both of these forgo brackets and closing tags in favor of indentation. Other developers prefer template languages like Handlebars and Liquid that look as close to traditional HTML as possible.

Four: Speed

The speed at which your site generator can build out each page becomes important as your content grows. A site with a handful of pages can take just milliseconds to build, whereas a site with archives of content can take several minutes to complete building. Because each change you publish will run the build process, speed is an important consideration.

Hugo is notoriously fast at building site pages, thanks to Go’s multithreaded architecture. Other site generators, by nature of their design goals, build both static HTML and a JavaScript bundle for each page. The JavaScript will be used to reduce load times for a user navigating around the site after it is on a CDN, but the extra work involved increases build time significantly.

Five: Developer Tooling

The core goal of a site generator is to transform content into HTML, but as you know, a site always contains other assets like CSS, images, and JavaScript. Modern frontend web development is starting to look a lot like traditional software development, with tool chains to prepare and compile assets for production.

Before publishing the site, images often need to be compressed, JavaScript transpiled and minified, and CSS combined or converted from formats such as SASS and LESS. Recently, it’s also become common to inline CSS and smaller images directly in the HTML for the performance benefit of the browser not needing to make an extra request to fetch these items.

So, the build of a JAMstack site is often both of these steps: creating the HTML and compiling these production assets. Some site generators concern themselves with only the HTML, leaving producing the rest of the assets to another utility like Gulp or Webpack. Other generators look to be more holistic and actually include features to handle compiling JavaScript CSS, and images—especially site generators built on JavaScript frameworks like Nuxt (Vuejs) or Gatsby (React). You’ll find a lot of generators actually make use of popular tools like Webpack under the hood.

Linting is also common. This is the process of automatically running tests against the HTML, JavaScript, and CSS to check for errors. This is a fantastic way to discover problems before they bite you in production. Linters are often particular about formatting, too, ensuring that everyone contributing adheres to the same standards.

Finally, developers working locally often will preview changes locally in a rapid-fire session of editing files and viewing changes in a browser. To support this, most site generators include a small web server that can allow you to serve the site locally for testing. Better yet is hot reloading, which is the ability for the browser to be updated immediately as a change is saved, with no need to click refresh or lose the current state of your application.

Recommending a site generator is a bit like recommending a computing device: it really depends on what you prefer, what you are familiar with, and how you intend to use it. There’s really no clear way to be prescriptive, unfortunately. Also, there is very active work being done in this space—by the time any “Best Of” list is published, it’s already out of date.

Luckily, trying a first run of any site generator usually takes only a few minutes. We really advise that you to try several and always browse the documentation for any design goals because those will inform you about the problems the creators are working to solve.

Adding Automation

Taking the time to add in build automation is what makes JAMstack development so productive and enjoyable.

All site generators have a build command that can be run from the terminal. The final assets are usually compiled to a folder named dist or build and then can be uploaded to almost any server or CDN. It’s certainly possible to build and then upload assets manually—especially early on. However, as you do more development and involve more team members, you’ll find this approach doesn’t scale.

We mentioned earlier that you can use a hosted Git repository for your project to provide an online backup and to facilitate collaboration. But now we explore the third reason: triggering automated builds via webhooks.

GitHub, GitLab, and Bitbucket are the three most popular repository hosting services and they all support webhooks. Webhooks are a simple way for any event on a platform (such as a code push) to trigger an HTTP POST to an external URL. When you set up your webhook, you can determine the URL used. The payload of the POST will contain JSON information about the event. The server receiving the webhook uses that information to determine what action to take.

For JAMstack sites, the most common action is to trigger a new build of the website when anything is added or modified via a push to the Git origin. Some more advanced deployment services will even build previews of pull requests at dedicated URLs. This way, teams can browse and test the changes before merging them. It’s also possible for each branch of a repository to be built separately at unique URLs. This is a powerful approach to staging environments and working on feature branches.

Ideally, your automated build process will do the following:

  1. Listen for notifications of changes to your Git repository (usually via webhooks).

  2. Prepare the build environment and fetch any required dependencies.

  3. Fetch remote data from APIs (as needed).

  4. Build the site and prepare assets.

  5. Publish the final site to a CDN.

Using GitHub Pages and GitLab Pages

If you are hosting a project’s repository at GitHub or GitLab, you might consider using GitHub Pages or GitLab Pages. Both are lightweight options that connect to your repositories and allow you to automatically build and host a static website. Both support custom domains.

GitHub Pages was originally launched to make developing sites and documentation for the open source projects they host easier. If you just need a simple website to explain your project, it’s a great option. Note that GitHub Pages supports only the Jekyll site generator, and only with specific plug-ins.

GitLab Pages supports a wider variety of static site generators, and you can even build your own continuous integration (CI) pipeline to deploy and host your website. GitLab also can be self-hosted if that’s a requirement or interest of your organization.

Using Your Own Infrastructure

If you have existing infrastructure, you might decide to run your own hosted build process on a server, virtual machine, or container. There are many tools from continuous integration/continuous development (CI/CD) that you can apply to automating the JAMstack. If you are already running CI/CD infrastructure, you might be able to plug JAMstack publishing into your existing workflows.

Using a Hosted Deployment Service

Because setting up a robust and resilient build system can be difficult, a new category of service has emerged: hosted build services/continuous integration. These services often have apps available for GitHub and GitLab, making the integration with your repository fairly painless. On some, you can select your own deployment target (like Amazon Web Services Simple Storage Service); on others, CDN and Domain Name System (DNS) services are baked in, providing everything you need to both build and host your application.

You want a build service that’s secure and customizable. Most build services will commonly create a temporary container, set up your environment, grab any needed dependencies, run the build, deploy the results, and then dispose of the container. It’s important that the build service is aware of and supports your site generator, language environment, and other tooling involved in building your website. You’ll want to keep dependency settings files like Gemfile, package.json, and composer.php accurate and up to date.

Selecting a CDN

Even with all the modern advancements in network technologies, performance for websites and applications still succumbs to the two oldest foes of the internet: time and space. Thankfully, the JAMstack approach of publishing directly to CDNs helps solve for both. First, the time required to start serving content is greatly improved by prerendering all the markup, as we discussed earlier. And second, the amount of physical distance (and network hops) that content must travel on the way to users is also reduced, thanks to the geographically distributed nature of CDNs.

A CDN is a global collection of edge nodes located in different data centers, all interconnected with fast, high-quality bandwidth. When a user loads content from a CDN, they are always connected to the closest edge node possible. And, of course, distributing your site across multiple edge nodes also helps availability. If one or more edge nodes, network interconnects, or even datacenters fail, your site remains available.

Here are three considerations for choosing a CDN for your web application:

Edge Locations

Publishing to a CDN usually means your site content becomes available to a minimum of 10 or 15 locations worldwide. You’ll want a CDN with edge locations close to your audience, so if you have users in Tokyo, they experience your website served from Tokyo.

Instant Updates

CDNs used to work as fairly unintelligent caching layers in front of web servers. Although they certainly sped up application performance, documents and images on the CDN were always given fairly lengthy Time To Live (TTL) settings. When cache was set to last hours or more, updates were delayed and purging the CDN to make way for new content proved rather difficult. It was often a delicate balance between allowing more dynamic content with frequent updates versus using longer caching to reduce the load on origin servers.

Modern CDNs now allow instant invalidation, replacing files in milliseconds and blurring the lines between static content and dynamic updates. If a near-instant update to CDN cache can be guaranteed each time the site is deployed, TTLs and stale content are no longer a concern. Today, rather than periodically pulling content from an origin server, updates can be pushed to CDNs each time the website changes. This advancement allows us to publish directly to CDNs and bypass servers—a critical component of the JAMstack.

Multiple Providers

Some CDNs locate their edge servers with multiple cloud providers, increasing the durability of your application. Even though a complete datacenter outage is rare, they do happen. It’s also not uncommon for a provider to experience a Distributed Denial-of-Service (DDoS) attack toward a particular facility or network. Ensure that your CDN provider is not only distributed across the globe but also across providers. Hosting on such a provider gives you the power of a “cloud of clouds” for the most resilient setup possible.

Providing Functionality: APIs

Even though a JAMstack site can indeed be nothing more than a collection of static assets, connecting your application to the vast world of API-based services is what shows the true power and promise of the JAMstack as an architecture for real-world applications.

It wasn’t too long ago that authentication, payment processing, and commerce were all concerns that you needed to host and manage yourself. But today, each of those (and many more features) are now consumable as APIs.

But how expensive will it be? What if the service goes down? How difficult would it be to rip out and replace? These questions are common when evaluating any hosted service, and it can feel daunting to trust critical services to a third party. But consider that all the questions above will also inevitably arise for teams that build or manage technologies in-house.

It all comes down to choice and flexibility, and the move toward decoupled, API-driven architectures (and away from monolithic apps) offers teams the flexibility to decide what to consume and what to build. That’s a powerful construct because each layer of the stack—from content, to commerce, to the frontend—can now be vetted and chosen independently.

Let’s look at that in action.

Consider the needs of an ecommerce store. First, the site needs to be fast and reliable, even under heavy load. The JAMstack’s ethos of static pages published to a global CDN ensures incredibly high marks in both speed and reliability. Without managing a fleet of servers, you can keep pace with the performance offered by the giants of ecommerce.

But equally important, you get full control over the entire frontend, down to the very last <div> and pixel. You are free to design and develop any page you can imagine, using the language, tools, and templates of your choosing. Your team can build experiences custom-tailored to your exact specifications. This is a very different approach from fighting the markup output by a traditional server-hosted ecommerce application.

Commerce, though, isn’t static. You’ll need a way to manage content (in this case, items for sale), power search, manage available inventory, and allow customers to make purchases. On the JAMstack, we use APIs to integrate services tailor-made for each component.

In the example flow shown in Figure 4-3, notice how content is managed in a hosted CMS service called Contentful. Store owners and editors use the Contentful interface to manage store listings and upload photos. Contentful is termed a “headless” CMS because its only concern is storing the content, not the display of it.

Sample JAMstack commerce setup
Figure 4-3. Sample JAMstack commerce setup

Adding a new item in Contentful triggers a new build of the website using Netlify’s build automation. During the build step, Netlify fetches the most recent store data from Contentful by calling the API. That data is fed into a site generator (like Gatsby) to render out each of the pages.

Also during the build step, APIs are used to send data to Algolia, the search provider, so the search index can be updated as the product catalog changes.

As visitors browse the site, prerendered pages are pulled directly from a CDN, with no running server process anywhere to be found. So how is the site made interactive? By taking advantage of the power of JavaScript to call APIs directly from the browser.

When a shopper types a search term, it’s fed to Algolia via Ajax, and Algolia returns JSON with the results. Javascript is used to display those results in whatever way we’d like to integrate them with our application.

After the shopper clicks the buy button, the intent to purchase is sent to a provider called CommerceLayer, whose job it is to power the shopping cart and checkout experience. Again, this happens using JavaScript to call APIs. Like Contentful, CommerceLayer is also headless and makes no demands on our how application looks and behaves.

There you have it: an entire commerce experience built by interconnecting modular components via API calls. Because there is no backend infrastructure to set up or manage, our entire time can be spent creating the frontend application. And because everything we’re using is a hosted service, a first prototype can be created in about a day or two.

This is why you might have heard of the JAMstack providing “superpowers” for frontend teams. Today, we can wire together rich, enterprise-grade functionality using API services like the ones here. But again, nothing is prescriptive on the JAMstack and there has been an explosion of new API service offerings designed to power just about any aspect of your project. In fact, you’ll find many traditional players like Shopify now offer APIs so they, too, can be consumed in a headless fashion. The explosion of APIs speaks to the momentum of the JAMstack as the architecture of the interconnected web.