Cloud Plumbing

When I titled this project To Pray Without Ceasing, I was serious. I want the system to be able to try to pray indefinitely for the needs expressed by Twitter users. Today I’ll share some of the challenges of making its unceasing prayer possible.

  1. TPWOC’s prayer-generation code outputs prayers in response to input texts gathered from Twitter, tweets that express needs (e.g. “I really need a friend”). Gathering these is easy enough. On a server, a program downloads recent examples of such tweets once a day via Twitter’s API. This is done automatically via a cron job.
  2. However, because it uses some large statistical models that must be loaded into memory, TPWOC’s prayer-generation code is too resource-hungry to run on this same server, and renting the requisite cloud-computing resources would likewise be prohibitively expensive. I want to make TPWOC sustainable (for Nokturno and for myself), so my current imperfect solution is simply to download the need-expressing tweets from my server and generate prayers for them using the resources of my local machine. Again using cron, I have set up my computer to try to do this automatically. It actually attempts to do so several times throughout the day in case I happen to have my computer turned off for some part of it. (Why not simply have Step 1 accomplished on my own computer rather than on my server? Because, due to limitations that Twitter places on its API, collecting tweets takes a rather long time. The more time-intensive processes that can be offloaded from my own computer, the better.)
  3. To make TPWOC’s prayer interface available on the web, I deploy it as a web-app via one of the major “cloud platform as a service” providers. But this web-app is simply a way of presenting the prayers, which it does not generate and which it does not at first possesses. I have set up my computer to periodically upload ready-to-pray prayers to an Amazon S3 “bucket,” a vessel from which my web-app can conveniently fetch them at regular intervals and display them to visitors.

The entire sequence of events looks like this:

I hardly expect these details to be particularly fascinating on their own. For my part, I do not relish setting up all this finicky pipework. However, I share these details in order to demonstrate that, if TPWOC is able to pray pseudo-infinitely, this effect is achieved not at the level of code per se but at the level of infrastructure—itself a medium that, like paint, film, or print, brings its particular affordances and limitations.