The Benefits of Combining Static Site Generation with Incremental Static Regeneration

Static site generation is the process of creating a website that consists of static HTML, CSS, and JavaScript files that are served directly to the user. This approach has many benefits, including faster load times, improved security, and lower hosting costs. However, it can be challenging to keep the content up-to-date without manually rebuilding the entire site. This is where incremental static site generation comes in. In this post, we will explore the benefits of combining static site generation with incremental static site generation.

Static Site Generation

Static site generation involves creating a website that consists of pre-built HTML, CSS, and JavaScript files. These files are generated once, and then served directly to users on request. This approach has many benefits, including faster load times, improved security, and lower hosting costs. However, the downside of static site generation is that it can be challenging to keep the content up-to-date. If you want to update the content on your website, you need to manually rebuild the entire site, which can be time-consuming.

Incremental Static Regeneration

Incremental static regeneration is a process that allows you to update your website's content without having to rebuild the entire site. Instead, the system only rebuilds the pages that have been updated, which can save a significant amount of time. By using incremental static site generation, you can keep your website's content up-to-date while still enjoying the benefits of static site generation.

Combining Static Site Generation with Incremental Static Regeneration

By combining static site generation with incremental static site generation, you can enjoy the benefits of both approaches. You can still benefit from the fast load times and improved security while also being able to update your website's content more easily. This approach is particularly useful for websites that have a lot of content that is updated frequently, such as news sites, blogs, and e-commerce sites.

A practical use case with Next.js

While recently re-working my website TheLastWeekIn.Dev which is built using Next.js, I opted to move the data from the markdown files stored locally onto PlanetScale as an opportunity to learn how PlanetScale works.

TheLastWeekIn.Dev post page

This meant that the previous approach of static site generation only would result in needing to fetch all the data every time there was a new build deployed for the website. As most of the static content did not change between builds, it would continue to (in my opinion) waste resources and eventually blow out build times.

Because of this, I opted to move towards incremental static regeneration. That being said, there was one problem with this: the next/link component was not respecting my fallback when prefetching. This meant hanging on the previous website for an non-insignificant, arbitrary amount of time while the user clicked.

Although a regenerated page is cached until revalidated on-demand (more on this later in the code examples), the first user for any yet-to-be generated page is in for a bad time.

As far as I am concerned, for this particular website, it’s okay if the new content goes through this process if the page has not been generated yet. What’s not okay is it happening for every page.

We can combat this by using getStaticPaths combined with our call to PlanetScale (or any database for that matter) to be smart about how we decide what pages to build on any given rebuild.

Smarter static site generation

On TheLastWeekIn.Dev, the route /[site]/blog/[tsx].tsx exposes the helper function getStaticPaths. This function determines what paths to statically generated at build time.

A modified version of this looks like so:

export const getStaticPaths: GetStaticPaths = async () => {
  const lastBuildTime = await prisma.staticSiteGeneration.findUnique({
    where: {
      environment: getEnvironment(process.env.NEXT_PUBLIC_VERCEL_ENV),
    },
    select: {
      updatedAt: true,
    },
  });

  const blogPosts = await prisma.blogPost.findMany({
    where: {
      updatedAt: {
        gte: lastBuildTime?.updatedAt ?? new Date(0),
      },
    },
  });

  // Update last attempted build time
  await prisma.staticSiteGeneration.upsert({
    where: {
      environment: getEnvironment(process.env.NEXT_PUBLIC_VERCEL_ENV),
    },
    update: {
      updatedAt: new Date(),
    },
    create: {
      updatedAt: new Date(),
      environment: getEnvironment(process.env.NEXT_PUBLIC_VERCEL_ENV),
    },
  });

  return {
    paths: blogPosts.map((blogPost) => ({
      params: {
        site: blogPost.publication.toString().toLowerCase(),
        slug: blogPost.edition.toString() ?? "1",
      },
    })),
    fallback: true,
  };
};

I am using Prisma as my ORM in this project, and so I have defined two important tables in my schema BlogPost and StaticSiteGeneration. The magic comes from the StaticSiteGeneration table.

Based on the environment that I am running (I’ve opted to use VERCEL_ENV to determine this as production, preview or development), I will check the last updatedAt value and only fetch the blog posts that have been built since the last timestamp. This means that if I have added 5 posts since the last build, it will only fetch those five posts and build those during this particular build. Any previously built page will still be cached thanks to ISR.

💡 As you might be able to tell, getStaticPaths looks to create a bit of a n+1 problem since data is not passed to getStaticProps. This post won’t address that, but there is an answer I am yet to look into that is meant to have an example using the file system cache.

Rebuilds and invalidations

For this particular website, since I want on-demand invalidation, I have set the revalidation to false within my getStaticProps function for that route.

💡 By default, if you don’t specify revalidate, it will be set to false. I am just doing it to be explicit.

If you run invalidation for a particular page, then Next.js will still serve the stale content on the next request, but regenerate the page and clear the cache in the background for you. It is some very cool tech.

You can set up your API to revalidate any given page from a particular route. Any example API looks like this:

import { z } from "zod";

import type { NextApiRequest, NextApiResponse } from "next";

const InvalidationSchema = z.object({
  secret: z.string(),
  url: z.string(),
});

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse
) {
  try {
    const { secret, url } = InvalidationSchema.parse(req.query);

    // Check for secret to confirm this is a valid request
    if (secret !== process.env.INVALIDATION_SECRET_TOKEN) {
      return res.status(401).json({ message: "Invalid token" });
    }

    await res.revalidate(url);
    return res.json({ revalidated: true });
  } catch (err) {
    // If there was an error, Next.js will continue
    // to show the last successfully generated page
    return res.status(500).send("Error revalidating");
  }
}

In my case, I am using zod to check the parameters passed are as expected (it will throw a 500 error when it is not which you can capture with an error monitoring tool like Sentry).

This API also only revalidates one URL at a time. You may want a solution that forces all of them to be revalidated in a case that all the pages must be rebuilt.

Alternatively, if you want your next build to statically generate pages that the timestamp on our StaticSiteGeneration timestamp is passed, you can simply reset the timestamp!

I have also set up another API route to do this:

import { captureException } from "@sentry/nextjs";
import { Environment, prisma } from "database";
import { z } from "zod";

import type { NextApiRequest, NextApiResponse } from "next";

const InvalidationSchema = z.object({
  secret: z.string(),
});
const EnvironmentSchema = z.nativeEnum(Environment);

const getEnvironment = (env: string | undefined) => {
  if (!env) {
    throw new Error("VERCEL_ENV is not set");
  }

  return EnvironmentSchema.parse(
    {
      development: Environment.DEVELOPMENT,
      production: Environment.PRODUCTION,
      preview: Environment.PREVIEW,
    }[env]
  );
};

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse
) {
  try {
    if (req.method !== "GET") {
      return res.status(405).json({ message: "Method not allowed" });
    }

    const { secret } = InvalidationSchema.parse(req.query);

    // Check for secret to confirm this is a valid request
    if (secret !== process.env.INVALIDATION_SECRET_TOKEN) {
      return res.status(401).json({ message: "Invalid token" });
    }

    const environment = getEnvironment(process.env.VERCEL_ENV);

    await prisma.staticSiteGeneration.update({
      where: {
        environment,
      },
      data: {
        updatedAt: new Date(0),
      },
    });
    return res.json({ revalidated: true });
  } catch (err) {
    captureException(err);
    // If there was an error, Next.js will continue
    // to show the last successfully generated page
    return res.status(500).send("Error revalidating");
  }
}

In the above code, you will not that I set the date with 0, which effectively means the next build will generate ALL webpages. You may want to alter this behaviour to manually set a date (in case you just want to rebuild an arbitary number of recent pages.

The above example also demonstrates usage with Sentry. You will obviously need to update the code based on your ORM and schema.

After you set the secret on your environment (in my case, through Vercel where the page is hosted), you can simply poll that endpoint at your discretion to force rebuilds or invalidations!

Conclusion

Static site generation and incremental static site generation are both useful approaches for building websites. By combining these two approaches, you can enjoy the benefits of both.

You can keep your website's content up-to-date without having to rebuild the entire site, while also enjoying the fast load times, improved security, and lower building costs of static site generation.

If you are building a website that has a lot of content that is updated frequently, then combining static site generation with incremental static site generation is definitely worth considering and an approach that enables both rebuilding and revalidating may be of interest to you!

In upcoming posts, I will be diving further into using Vercel, Next.js, PlanetScale and Prisma specifically for your applications, so be sure to subscribe if you like the content!