An LLM-Turbocharged Community Calendar Reboot

Back in the 2010s I created a system to aggregate public calendars for cities and towns. The Elm City project, as it was called, never gained traction, but the problem it addressed remains unsolved. Calendars of public events are information silos; there’s still no good way to combine them. I wondered how effectively my team of LLM assistants could help me reboot that community-scale calendar aggregator. It turned out to be a shockingly fast speed run; here are the highlights.

RSS for Calendars

The inspiration for the original project was the blogosphere: a network of RSS publishers, readers, and aggregators. Why not build an analogous network where publishers, readers, and aggregators exchange iCalendar feeds? The poster child for this idea was Google Calendar. The YMCA, the public library, or the city government could publish event calendars by embedding Google Calendar on their sites; community members could subscribe to them in Google Calendar; aggregators could combine them using the iCalendar URLs that Google Calendar provides. It wasn’t all about Google Calendar, though. Any online calendar could provide iCalendar feeds too, and many (though by no means all) did. Failing that, it was possible (though arduous) to scrape web calendars and convert them into feeds that an iCalendar-based aggregator could read.

With the help of Claude and ChatGPT, it was trivial to read iCalendar URLs and transform them to lists of Python dicts representing events.

The original project, sponsored by Microsoft, used Azure, .NET, and C#. I was mildly curious to see how well LLMs can support those technologies, but you get the smoothest ride with Python and JavaScript, so I was happy to hitch that ride this time around. And iCalendar, the Python library for parsing and generating iCalendar data, has been maturing for 20 years. With the help of Claude and ChatGPT, it was trivial to read iCalendar URLs and transform them to lists of Python dicts representing events.

Wrangling Timezones

It was less trivial to work out how to recognize the various methods of encoding time zone information, and to normalize events to the local time observed by a community. That had originally entailed plenty of head-scratching and trial-and-error, as anyone who has ever wrestled with time zones will appreciate. Along the way, I wrote an essay, A literary appreciation of the Olson/Zoneinfo/tz database, that captures some of the craziness. This time, my team and I converged much more quickly on a solution. I asked Claude to document the strategy, and here (with minor editing) is part of what it wrote.

The strategy is pretty much the same as I’d settled on last time. Handling the “experimental” X-WR-TIMEZONE property (scare quotes because it’s used pretty widely) wasn’t my idea, ChatGPT suggested that. It rang a bell, though, and when I searched I found my own article “X-WR-TIMEZONE considered harmful?” on the first page of results! Having walked that path before I’d have come to the same conclusion again, but as with all aspects of this project, the LLM assist accelerated the process.

A Static HTML Rendering of Aggregated Calendars

People like to see calendar data in a calendar layout. Although I knew the project would also require a list view, because a busy community calendar will overwhelm a calendar layout, I decided to start with a calendar layout. There are, of course, many calendar widgets to choose from. But I wanted to keep things brutally simple: no dependencies, just vanilla HTML/CSS if possible.

Here’s the solution Claude came up with.

It relies on Jinja2 templates, about which I knew only a little. Claude’s solution not only worked straightaway for the calendar view, but then served double duty when I got around to implementing the complementary list view.

I’m really pleased with it! I’ve used a variety of calendar widgets over the years — they’ve always required dependencies of one form or another and weren’t as easy to customize as this beautifully simple approach.

I continue to marvel at the way in which this style of work provides just-in-time learning opportunities. In the Before Time, if I needed to learn about Jinja2 templating, I’d have struggled to bridge the gap between the available documentation and the actual code I was wrangling. By tailoring that documentation to your live context, LLMs deliver a far more effective learning experience.

Hosting the Calendars

Once I could parse calendar feeds, combine them, and render them as static HTML pages, I needed a way to host those pages. This was an opportunity to learn about GitHub Pages, a feature of GitHub that I’d tire-kicked in the past but never actually used. It’s gotten more complicated since the last time I looked, when there was only the branch-based (“traditional”) method. Now there’s also a more advanced custom-workflow-based method.

Here was my requirement. I wanted a structure like so:

/root/santarosa/ 2024-09.html 2024-10.html /root/bloomington 2024-09.html 2024-10.html

And I wanted to serve those pages at URLs like SOMETHING/santarosa/2024-09.html. I was pretty sure I wanted to stick with the simpler branch-based method but I kept running into 404 errors. Eventually, I figured out that, even though I didn’t need or want an index.html in those directories — because I was only planning to link to the calendar pages — GitHub Pages evidently requires an index.html in order to activate a directory to serve pages within it.

When I discovered that, I asked ChatGPT explicitly if it’s OK to serve pages from a directory with no index.html and it chirped “Certainly, it’s perfectly fine to omit index.html”. Nope!

Will that memory kick in when I next have need of it, perhaps years from now? It seems unlikely, but I do like the idea that LLMs can at least try to remember corrections. Have any of the other corrections I’ve stored in ChatGPT’s memory been helpful? As with everything else about these slippery creatures, it’s hard to know for sure.

Synthesizing Calendar Feeds

This “RSS for calendars” system presumes that online calendars make iCalendar feeds available for aggregation. But often they don’t. And sadly, such feeds are rarer now than they were during the first iteration of this project. Back then, I considered but rejected the idea of scraping web calendars to synthesize iCalendar feeds. It was doable but impractical. The problem with web scrapers isn’t so much that they can break when page structures change. In practice that doesn’t happen very often; once you’ve got a working scraper it tends to keep working. But writing a scraper is a finicky and time-consuming chore. So the Elm City project sought to teach people how and why to publish data in structured formats that could support automatic syndication, and could also be repurposed to create web pages in any desired style. Google Calendar does that for you: it can render your calendar as both a feed and an interactive widget. Alternate renderings from any iCalendar source are eminently feasible.

Suppose you manage the calendar of public events for your YMCA, public library, or city council. It’s natural to manage those calendars using tools like Google Calendar or Outlook. Now you want to promote those public events in a community calendar. When you go there, you’re confronted with the still-ubiquitous “Submit an event” workflow which requires you to retype all the same facts you just typed into your calendar program. Even worse, you may want to promote your events on more than one public calendar, so you have to retype those facts again for each of them. But there’s no need for that! Your Google, Microsoft, or other calendar program can provide a feed, and one or more public calendars can consume that feed. Events could, and should, flow frictionlessly in an iCalendar-based publish/subscribe network.

That was the dream, and maybe it’ll come true someday, but for now, public calendars are bespoke information silos that serve web pages in all kinds of random ways. Those pages meet people’s needs pretty well, but thwart machines looking for the information structure implicit within them. Or anyway, that’s how things were until recently. But as I discovered in How LLMs Can Unite Analog Event Promotion and Digital Calendars, LLMs blur the boundary between unstructured and structured data. In that column, I focused on their remarkable ability to turn the images on event posters into iCalendar data. But as Simon Willison shows in his stunning datasette-extract demo, they can also pull structured data from semi-structured webpages.

It would be insanely expensive to use LLMs to convert calendar pages to iCalendar feeds at scale, [but] you can use them to write web scrapers that are cheap to run and easy to verify.

It would be insanely expensive to use LLMs to convert calendar pages to iCalendar feeds at scale, and you’d never know if they got everything exactly right. But you can also use them to write web scrapers that are cheap to run and easy to verify. Using LLMs to do that is a game-changer. I can capture the fragment of HTML that represents an event in a particular rendering, hand that to ChatGPT or Claude along with an example of an existing scraper, and arrive at a new scraper that in some cases works right away, or in others requires only minor tweaks.

The LLMs handle the mind-numbing details: figuring out the selectors to use in JavaScript or in Python’s BeautifulSoup, parsing dates and times presented in all sorts of ways, matching HTML elements to the right slots in the corresponding iCalendar schema, navigating multi-page views, implementing backoff/retry when required. Writing all that code for many different kinds of web calendars would entail a crazy level of effort. Delegating that work can make this admittedly undesirable approach at least feasible.

Two major patterns have emerged. Some sites are amenable to Python scripting, either because they do server-side rendering or because they include JavaScript that makes API calls Python can leverage. For others that are too JavaScript-intensive for Python to deal with, there’s the potent combination of a headless browser and Puppeteer (or, if you prefer, Playwright). I’d previously climbed Puppeteer’s learning curve with difficulty. This time around I got up to speed very quickly with Node-based scripts that could scrape iCalendar feeds from Python-resistant web calendars. Here too, I paused to marvel at the power of LLM-assisted learning. My original Puppeteer-based projects required hours of research and trial-and-error. Nowadays you can speedrun that process in minutes.

Do the Same Thing a Different Way

The original Elm City project was dedicated to the proposition that people who publish structured information should use tools that grok the structure and comply with the Internet standards that define it. I still think that’s right. If you’re publishing a calendar to the web, the information should probably originate in an iCalendar-aware system. That approach is eminently possible, I encourage it, and this reboot of the project happily ingests calendars from any standard source.

It’s not a perfect solution, but with LLM assistance it’s a workable one.

But the majority of web calendars don’t support the standard, and people who have adopted tools quite reasonably don’t like to be told they ought to be using different ones. Last time around I tried and failed to overcome that resistance. In fairness, there wasn’t a workable alternative. Now that there is, let’s not die on that hill. Communities that want to build comprehensive public calendars will be able to do so using a hybrid approach that blends existing iCalendar feeds with feeds synthesized from web calendars. It’s not a perfect solution, but with LLM assistance it’s a workable one. And who knows, maybe if people see what’s possible when information silos converge, the common tools that can ease convergence will seem more attractive.