Getting Started
If you want to process an RSS feed programmatically, you have to run code to poll the feed and keep track of items already processed. This isn't hard to write, but it's often not core to your app's logic. You probably just want to run code on each new item in the feed.
// Don't care how I get feed
for (const item of feed) {
// Just want to run code
}
Pipedream lets you treat an RSS feed as anevent source. Pipedream runs the code to poll the feed, emitting new items as the feed produces them:
Then, you can process items from your event source in 3 ways:
To get started,create a new RSS source in the Pipedream UI. Name the source and add your feed URL:
You can also visithttps://pipedream.com/sourcesand click theCreate Sourcebutton, then choose the rss/rss.jssource from the modal that appears.
If you don't have a feed handy, you can use https://rss.m.pipedream.net. This generates a new, random item once every 15 minutes.
You can also create an event source using the Pipedream CLI:
$ curl https://cli.pipedream.com/install | sh
$ pd deploy # Select the rss source, enter your URL
Once deployed, you can retrieve items emitted by the source using the pd eventscommand:
$ pd events -n 1 sample-feed# Retrieve the newest item
$ pd events -f sample-feed# Tail the feed in real time
Then, you can trigger aPipedream workflow— a serverless Node.js script — on every event:
or process items in your own app usingthe SSE stream or REST APItied to your source.
Example Workflows
SSE and REST APIs
Pipedream provides two other interfaces for accessing events produced by sources:
This way, you can run an event source in Pipedream but access its events in your own application.
SSE
SSE —Server-sent events—defines a spec for how servers can send events directly to clients that subscribe to those events, similar toWebSockets and related server-to-client push technologies. If you listen for new items using SSE, you can run any code in your own application, in real time, as soon as those items are added to the feed.
Streams are private by default. You authenticate by passing your Pipedream API key in the Authorization header:
$ curl -H "Authorization: Bearer API_KEY" \
https://api.pipedream.com/sources/SOURCE_ID/sse
See the Pipedream SSE docs for more information on authentication, example Node code, and other details on the interface.
REST API
If you prefer to process items from an RSS feed in batch, you can fetch them using Pipedream's REST API:
$ curl -H "Authorization: Bearer API_KEY" \
https://api.pipedream.com/v1/sources/SOURCE_ID/events?n=1
Note the?n=1query string. You can vary the number of events returned (most recent first) by setting this param.