Any guesses on what we will recommend here? YOU GUESSED IT! CircuitBuilder!
TLDR; WANT THIS CIRCUIT OR ONE LIKE IT? SEND US AN EMAIL.
Let's start with a story. This morning, a colleague of ours sent us a text, asking for simple solutions we could provide for someone he knows who is an employment attorney. She is having to keep up with changes to legislation in her state so that she can properly advise her clients. The problem is that the web site that publishes the current legislation updates multiple times a day, and there are no notifications for when it updates. She wants a notification when it has updated, and ideally, to know what changed.
As software developers, we know that there are a million services and libraries for handling this kind of thing. But what is an attorney to do? In this exact case, call us. (spoiler alert! We had her level 1 solution in place within 20 minutes!)
Let's walk through what we did for her. The first, and simplest solution was to create a circuit to check the site, and then notify someone each time the site had a modification. This looks like a Trigger (or a timer, or both), an HTTP Request, a Modified, and then a Text message (in this case, to me and to her, so I could follow along).
This is the simplest possible case. It took about 3 minutes to configure. And to be fair, it is oversimplified. Want to know what actually happened and how we solved it? Sure, let's dig deeper.
The html in that request actually CHANGES all the time, on just ONE LINE, in a script tag. Investigating, this was not something that changed the content of the page. Curious? This was the line (line 65 in the output from the HTTP Request): https://gist.github.com/dawnTestCode/81cba3c6acd82bd02899b6c5f1076134. The exact changing text looked like this (I used Visual Studio Code to show me)
After determining that the characters that were changing did not impact the relevant content, we decided to drop in a little code component to filter out those script tags. A little Google fu later to find some javascript, and tada! Here is our code (click it for a link to a gist of the code).
![var bodytext = ''; var m = value.data.match(/]*>([^<]*(?:(?!<\/?body)<[^<]*)*)<\/body\s*>/i); if (m) bodytext = m[1]; console.log(bodytext) send(0, bodytext);](http://custom-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_9000,w_1200,f_auto,q_auto/1211129/347256_985146.png "var bodytext = ''; var m = value.data.match(/]*>([^<]*(?:(?!<\/?body)<[^<]*)*)<\/body\s*>/i); if (m) bodytext = m[1]; console.log(bodytext) send(0, bodytext);")
After verifying that that worked, we set the timer to run every 30 minutes. The whole thing took us about 20 minutes to implement.
Want to know the super secret, undisclosed part? We have this circuit running in our super-alpha-version of CircuitBuilder's CloudRunner. Up in the cloud, it is running every 30 minutes. We put it up there around 10:45am today, and the web site has changed once as of writing of this post (1:49pm), at 12:18pm.
Finally, that only covers the notification part. Next is another post describing how we have chosen to handle notifying her of WHAT has changed.