1
0

10 Commits

Author SHA1 Message Date
George Mandis
ffdaf0b72b 2.0.2 2023-06-03 15:28:41 -04:00
George Mandis
5ebc0d4952 Fixing a bug where failing to fetch a feed could make Bubo hang 2023-06-03 15:28:29 -04:00
George Mandis
04696e9df6 Updating packages 2023-06-03 15:28:10 -04:00
George Mandis
0fa785b41d Update README.md 2022-12-04 20:16:50 -05:00
George Mandis
0fbe6c16ad Updates for package, node and README (#12) 2022-12-04 17:49:50 -05:00
George Mandis
bdb8bf8ef4 Updated demo/default feeds list (#11) 2022-12-04 17:37:31 -05:00
George Mandis
adb1227b95 Update README.md 2022-12-04 17:33:17 -05:00
dependabot[bot]
e9f570ab07 Bump node-fetch from 3.1.0 to 3.1.1 (#7)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-31 11:48:44 -07:00
Antoni
7e9d187d37 Fixed typo in README.md (#9) 2022-08-31 11:48:32 -07:00
Marko Vujanic
58fa8d0d47 Fix typo de;ay -> delay (#8) 2022-07-06 08:25:02 -04:00
6 changed files with 1130 additions and 1023 deletions

2
.nvmrc
View File

@@ -1 +1 @@
16.4.1 18.12.1

View File

@@ -1,36 +1,28 @@
[![Netlify Status](https://api.netlify.com/api/v1/badges/81dd219c-51cb-4418-a18c-42c8b104c689/deploy-status)](https://app.netlify.com/sites/bubo-rss-demo/deploys) [![Netlify Status](https://api.netlify.com/api/v1/badges/81dd219c-51cb-4418-a18c-42c8b104c689/deploy-status)](https://app.netlify.com/sites/bubo-rss-demo/deploys)
# 🦉 Bubo Reader (2.0) # 🦉 Bubo Reader
Bubo Reader is a hyper-minimalist <acronym title="Really Simple Syndication">RSS</acronym> and <acronym title="JavaScript Object Notation">JSON</acronym> feed reader you can deploy on your own server, [Netlify](https://netlify.com) in a few steps or [Glitch](https://glitch.com) in even fewer steps! The goal of the project is to generate a webpage that shows a list of links from a collection of feeds organized by category and website. That's it. Bubo Reader is a hyper-minimalist feed reader (RSS, Atom, JSON) you can deploy on your own server, [Netlify](https://netlify.com) in a few steps or [Glitch](https://glitch.com) in even fewer steps! The goal of the project is to generate a webpage that shows a list of links from a collection of feeds organized by category and website. That's it.
It is named after this [silly robot owl](https://www.youtube.com/watch?v=MYSeCfo9-NI) from Clash of the Titans (1981). It is named after this [silly robot owl](https://www.youtube.com/watch?v=MYSeCfo9-NI) from Clash of the Titans (1981).
You can read more about how this project came about on my blog: You can read more about this project on my blog:
- [Introducing Bubo RSS: An Absurdly Minimalist RSS Feed Reader](https://george.mand.is/2019/11/introducing-bubo-rss-an-absurdly-minimalist-rss-feed-reader/). - [Introducing Bubo RSS: An Absurdly Minimalist RSS Feed Reader](https://george.mand.is/2019/11/introducing-bubo-rss-an-absurdly-minimalist-rss-feed-reader/).
- [Publishing Bubos RSS to Netlify with GitHub Actions](https://george.mand.is/2020/02/publishing-bubos-rss-to-netlify-with-github-actions/)
## Get Started
## Getting Started - Clone or fork the repo and run `npm install` to install the dependencies.
- Clone or fork the repo and run `npm install` to install the dependencies.
- Update `feeds.json` to include categories and links to feeds you would like to see. - Update `feeds.json` to include categories and links to feeds you would like to see.
- Run `npm run build:bubo` - Run `npm run build:bubo`
That's it! You should now have a static page with links to the latest content from your feeds in the `public` folder, ready to serve. That's it! You should now have a static page with links to the latest content from your feeds in the `public` folder, ready to serve.
## Differences in Bubo 2.0 <details>
<summary>
Version 2.0 has introduced some substantial changes for Bubo! While the static output remains endearingly spartan, the engine that builds it has changed a bit. <strong>Anatomy of Bubo Reader</strong>
</summary>
Hopefully all of these changes are in services of making this project more useful to others and encouraging outside contributions.
Changes of note:
- Bubo has been rewritten in [TypeScript](https://www.typescriptlang.org/). It's pretty slick! I anticipate the typing could be improved, but it's a start.
- You fill find an `.nvmrc` file in the root of this project. Learn more [about nvm](https://github.com/nvm-sh/nvm) if you're unfamiliar.
- The script will actually write your `index.html` file for you (Previously the build script simply ran `node src/index.js > output/index.html`). It makes a strong assumption that this file lives in the `public` folder.
- There is a somewhat sophisticated mechansim in-place for batching & throttling your requests, if needed.
## Anatomy of Bubo Reader
The static pieces: The static pieces:
@@ -45,14 +37,19 @@ The engine:
- `src/renderer.ts` — The renderer that loads Nunjucks, the template and understands how to process the incoming feed data. Prefer something else? This is the place to change it! - `src/renderer.ts` — The renderer that loads Nunjucks, the template and understands how to process the incoming feed data. Prefer something else? This is the place to change it!
- `src/utilities.ts` — A variety of parsing and normalization utilities for Bubo, hidden away to try and keep things clean. - `src/utilities.ts` — A variety of parsing and normalization utilities for Bubo, hidden away to try and keep things clean.
## Throttling </details>
In the main `index.ts` file you will find two values that allow yout to batch and throttle your feed requests: <details>
<summary>
<strong>Throttling</strong>
</summary>
- `MAX_CONNECTIONS` dictates the maximium number of requests a batch can have going at once. In the main `index.ts` file you will find two values that allow you to batch and throttle your feed requests:
- `DELAY_MS` dictates the amount of de;ay time between each batch.
The default configuration is **no batching or throttling** beacuse `MAX_CONNECTIONS` is set to `Infinity`. If you wanted to change Bubo to only fetch one feed at a time every second you could set these values to: - `MAX_CONNECTIONS` dictates the maximum number of requests a batch can have going at once.
- `DELAY_MS` dictates the amount of delay time between each batch.
The default configuration is **no batching or throttling** because `MAX_CONNECTIONS` is set to `Infinity`. If you wanted to change Bubo to only fetch one feed at a time every second you could set these values to:
```javascript ```javascript
const MAX_CONNECTIONS = 1; const MAX_CONNECTIONS = 1;
@@ -67,22 +64,20 @@ const DELAY_MS = 2500;
``` ```
In practice, I've never _really_ run into an issue leaving `MAX_CONNECTIONS` set to `Infinity` but this feels like a sensible safeguard to design. In practice, I've never _really_ run into an issue leaving `MAX_CONNECTIONS` set to `Infinity` but this feels like a sensible safeguard to design.
## Demos
You can view live demos here: </details>
- [https://bubo-rss-demo.netlify.com/](https://bubo-rss-demo.netlify.com/) <details>
- [http://bubo-rss.glitch.me/](http://bubo-rss.glitch.me/) <summary>
<strong>Getting Started</strong>
Not the most exciting-looking demos, I'll admit, but they work! </summary>
**Getting Started**
- [Deploying to Glitch](#glitch) - [Deploying to Glitch](#glitch)
- [Deploying to Netlify](#netlify) - [Deploying to Netlify](#netlify)
- [Keeping feeds updated](#updated) - [Keeping feeds updated](#updated)
<a id="glitch"></a> <a id="glitch"></a>
## Deploying to Glitch ## Deploying to Glitch
The quickest way is to remix the project on Glitch: The quickest way is to remix the project on Glitch:
@@ -93,6 +88,7 @@ There is also a `glitch` branch on this repo if you'd prefer to start there.
Just change some feeds in `./config/feeds.json` file and you're set! If you'd like to modify the style or the template you can changed `./public/style.css` file or the `./config/template.html` file respectively. Just change some feeds in `./config/feeds.json` file and you're set! If you'd like to modify the style or the template you can changed `./public/style.css` file or the `./config/template.html` file respectively.
<a id="netlify"></a> <a id="netlify"></a>
## Deploying to Netlify ## Deploying to Netlify
- [Fork the repository](https://github.com/georgemandis/bubo-rss/fork) - [Fork the repository](https://github.com/georgemandis/bubo-rss/fork)
@@ -102,6 +98,7 @@ Just change some feeds in `./config/feeds.json` file and you're set! If you'd li
The deploy settings should automatically import from the `netlify.toml` file. All you'll need to do is confirm and you're ready to go! The deploy settings should automatically import from the `netlify.toml` file. All you'll need to do is confirm and you're ready to go!
<a id="updated"></a> <a id="updated"></a>
### Keeping Feeds Updated ### Keeping Feeds Updated
#### Using Netlify Webhooks #### Using Netlify Webhooks
@@ -112,21 +109,29 @@ To keep your feeds up to date you'll want to [setup a Build Hook](https://www.ne
- [Zapier](https://zapier.com/) - [Zapier](https://zapier.com/)
- [EasyCron](https://www.easycron.com/) - [EasyCron](https://www.easycron.com/)
#### Using GitHub Actions
Coming soon—there is an old branch that demonstrates this, but it needs to be revisisted in light of Bubo 2.0.
#### Rolling Your Own #### Rolling Your Own
If you already have a server running Linux and some command-line experience it might be simpler to setup a [cron job](https://en.wikipedia.org/wiki/Cron). If you already have a server running Linux and some command-line experience it might be simpler to setup a [cron job](https://en.wikipedia.org/wiki/Cron).
</details>
## Demos
You can view live demos here:
- [https://bubo-rss-demo.netlify.com/](https://bubo-rss-demo.netlify.com/)
- [http://bubo-rss.glitch.me/](http://bubo-rss.glitch.me/)
## Support ## Support
If you found this useful please consider sponsoring me or this project. If you found this useful please consider [sponsoring me or this project](https://github.com/sponsors/georgemandis).
If you'd rather run this on your own server please consider using one of these affiliate links to setup a micro instance on [Linode](https://www.linode.com/?r=8729957ab02b50a695dcea12a5ca55570979d8b9), [Digital Ocean](https://m.do.co/c/31f58d367777) or [Vultr](https://www.vultr.com/?ref=8403978). If you'd rather run this on your own server please consider using one of these affiliate links to setup a micro instance on [Linode](https://www.linode.com/?r=8729957ab02b50a695dcea12a5ca55570979d8b9), [Digital Ocean](https://m.do.co/c/31f58d367777) or [Vultr](https://www.vultr.com/?ref=8403978).
## Showcase ## Showcase
Here are some websites using Bubo Reader: Here are some websites using Bubo Reader:
- [Kevin Fiol](https://kevinfiol.com/reader/) ([repo](https://github.com/kevinfiol/reader)) - [Kevin Fiol](https://kevinfiol.com/reader/) ([repo](https://github.com/kevinfiol/reader))
Please share if you would like to be featured!

View File

@@ -1,16 +1,20 @@
{ {
"Web Development": [ "Developer News": [
"https://hacks.mozilla.org/feed/", "https://hacks.mozilla.org/feed/",
"https://web.dev/feed.xml", "https://web.dev/feed.xml",
"https://v8.dev/blog.atom", "https://v8.dev/blog.atom",
"https://alistapart.com/main/feed/", "https://alistapart.com/main/feed/",
"https://css-tricks.com/feed/", "https://css-tricks.com/feed/",
"https://dev.to/feed" "https://dev.to/feed",
"https://changelog.com/feed"
], ],
"Blogs": [ "Blogs": [
"https://george.mand.is/feed.xml", "https://george.mand.is/feed.xml",
"https://joy.recurse.com/feed.atom" "https://joy.recurse.com/feed.atom"
], ],
"Social": [
"https://social.mandis.dev/@georgemandis.rss"
],
"My GitHub Projects": [ "My GitHub Projects": [
"https://github.com/georgemandis.atom", "https://github.com/georgemandis.atom",
"https://github.com/georgemandis/bubo-rss/releases.atom", "https://github.com/georgemandis/bubo-rss/releases.atom",

1997
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{ {
"name": "bubo-reader", "name": "bubo-reader",
"version": "2.0.1", "version": "2.0.2",
"description": "A simple but effective feed reader (RSS, JSON)", "description": "A simple but effective feed reader (RSS, JSON)",
"homepage": "https://github.com/georgemandis/bubo-rss", "homepage": "https://github.com/georgemandis/bubo-rss",
"main": "src/index.js", "main": "src/index.js",
@@ -27,19 +27,19 @@
}, },
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"chalk": "^5.0.0", "chalk": "^5.2.0",
"node-fetch": "^3.1.0", "node-fetch": "^3.3.1",
"nunjucks": "^3.2.0", "nunjucks": "^3.2.4",
"rss-parser": "^3.6.3" "rss-parser": "^3.13.0"
}, },
"devDependencies": { "devDependencies": {
"@types/node": "^16.11.10", "@types/node": "^16.18.34",
"@types/nunjucks": "^3.2.0", "@types/nunjucks": "^3.2.2",
"@types/xml2js": "^0.4.9", "@types/xml2js": "^0.4.11",
"@typescript-eslint/eslint-plugin": "^5.4.0", "@typescript-eslint/eslint-plugin": "^5.59.8",
"@typescript-eslint/parser": "^5.4.0", "@typescript-eslint/parser": "^5.59.8",
"eslint": "^8.3.0", "eslint": "^8.42.0",
"tslib": "^2.3.1", "tslib": "^2.5.3",
"typescript": "^4.5.2" "typescript": "^4.9.5"
} }
} }

View File

@@ -66,7 +66,11 @@ let completed = 0;
* and we want to build the static output. * and we want to build the static output.
*/ */
const finishBuild: () => void = async () => { const finishBuild: () => void = async () => {
console.log("\nDone fetching everything!"); completed++;
// if this isn't the last feed, just return early
if (completed !== feedListLength) return;
process.stdout.write("\nDone fetching everything!\n");
// generate the static HTML output from our template renderer // generate the static HTML output from our template renderer
const output = render({ const output = render({
@@ -77,10 +81,10 @@ const finishBuild: () => void = async () => {
// write the output to public/index.html // write the output to public/index.html
await writeFile("./public/index.html", output); await writeFile("./public/index.html", output);
console.log( process.stdout.write(
`\nFinished writing to output:\n- ${feedListLength} feeds in ${benchmark( `\nFinished writing to output:\n- ${feedListLength} feeds in ${benchmark(
initTime initTime
)}\n- ${errors.length} errors` )}\n- ${errors.length} errors\n`
); );
}; };
@@ -103,8 +107,7 @@ const processFeed =
}) => }) =>
async (response: Response): Promise<void> => { async (response: Response): Promise<void> => {
const body = await parseFeed(response); const body = await parseFeed(response);
completed++; //skip to the next one if this didn't work out
// skip to the next one if this didn't work out
if (!body) return; if (!body) return;
try { try {
@@ -124,20 +127,19 @@ const processFeed =
}); });
contentFromAllFeeds[group].push(contents as object); contentFromAllFeeds[group].push(contents as object);
console.log( process.stdout.write(
`${success("Successfully fetched:")} ${feed} - ${benchmark(startTime)}` `${success("Successfully fetched:")} ${feed} - ${benchmark(startTime)}\n`
); );
} catch (err) { } catch (err) {
console.log( process.stdout.write(
`${error("Error processing:")} ${feed} - ${benchmark( `${error("Error processing:")} ${feed} - ${benchmark(
startTime startTime
)}\n${err}` )}\n${err}\n`
); );
errors.push(`Error processing: ${feed}\n\t${err}`); errors.push(`Error processing: ${feed}\n\t${err}`);
} }
// if this is the last feed, go ahead and build the output finishBuild();
completed === feedListLength && finishBuild();
}; };
// go through each group of feeds and process // go through each group of feeds and process
@@ -150,15 +152,16 @@ const processFeeds = () => {
for (const feed of feeds) { for (const feed of feeds) {
const startTime = Date.now(); const startTime = Date.now();
setTimeout(() => { setTimeout(() => {
console.log(`Fetching: ${feed}...`); process.stdout.write(`Fetching: ${feed}...\n`);
fetch(feed) fetch(feed)
.then(processFeed({ group, feed, startTime })) .then(processFeed({ group, feed, startTime }))
.catch(err => { .catch(err => {
console.log( process.stdout.write(
error(`Error fetching ${feed} ${benchmark(startTime)}`) error(`Error fetching ${feed} ${benchmark(startTime)}\n`)
); );
errors.push(`Error fetching ${feed} ${err.toString()}`); errors.push(`Error fetching ${feed} ${err.toString()}\n`);
finishBuild();
}); });
}, (idx % (feedListLength / MAX_CONNECTIONS)) * DELAY_MS); }, (idx % (feedListLength / MAX_CONNECTIONS)) * DELAY_MS);
idx++; idx++;