1
0

12 Commits

Author SHA1 Message Date
George Mandis
78629d2757 Updated packages 2023-06-03 15:32:13 -04:00
George Mandis
b450b6ed54 Removed old social media feed 2023-06-03 15:31:34 -04:00
George Mandis
ffdaf0b72b 2.0.2 2023-06-03 15:28:41 -04:00
George Mandis
5ebc0d4952 Fixing a bug where failing to fetch a feed could make Bubo hang 2023-06-03 15:28:29 -04:00
George Mandis
04696e9df6 Updating packages 2023-06-03 15:28:10 -04:00
George Mandis
0fa785b41d Update README.md 2022-12-04 20:16:50 -05:00
George Mandis
0fbe6c16ad Updates for package, node and README (#12) 2022-12-04 17:49:50 -05:00
George Mandis
bdb8bf8ef4 Updated demo/default feeds list (#11) 2022-12-04 17:37:31 -05:00
George Mandis
adb1227b95 Update README.md 2022-12-04 17:33:17 -05:00
dependabot[bot]
e9f570ab07 Bump node-fetch from 3.1.0 to 3.1.1 (#7)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-31 11:48:44 -07:00
Antoni
7e9d187d37 Fixed typo in README.md (#9) 2022-08-31 11:48:32 -07:00
Marko Vujanic
58fa8d0d47 Fix typo de;ay -> delay (#8) 2022-07-06 08:25:02 -04:00
6 changed files with 1128 additions and 1024 deletions

2
.nvmrc
View File

@@ -1 +1 @@
16.4.1
18.12.1

View File

@@ -1,16 +1,17 @@
[![Netlify Status](https://api.netlify.com/api/v1/badges/81dd219c-51cb-4418-a18c-42c8b104c689/deploy-status)](https://app.netlify.com/sites/bubo-rss-demo/deploys)
# 🦉 Bubo Reader (2.0)
# 🦉 Bubo Reader
Bubo Reader is a hyper-minimalist <acronym title="Really Simple Syndication">RSS</acronym> and <acronym title="JavaScript Object Notation">JSON</acronym> feed reader you can deploy on your own server, [Netlify](https://netlify.com) in a few steps or [Glitch](https://glitch.com) in even fewer steps! The goal of the project is to generate a webpage that shows a list of links from a collection of feeds organized by category and website. That's it.
Bubo Reader is a hyper-minimalist feed reader (RSS, Atom, JSON) you can deploy on your own server, [Netlify](https://netlify.com) in a few steps or [Glitch](https://glitch.com) in even fewer steps! The goal of the project is to generate a webpage that shows a list of links from a collection of feeds organized by category and website. That's it.
It is named after this [silly robot owl](https://www.youtube.com/watch?v=MYSeCfo9-NI) from Clash of the Titans (1981).
You can read more about how this project came about on my blog:
You can read more about this project on my blog:
- [Introducing Bubo RSS: An Absurdly Minimalist RSS Feed Reader](https://george.mand.is/2019/11/introducing-bubo-rss-an-absurdly-minimalist-rss-feed-reader/).
- [Publishing Bubos RSS to Netlify with GitHub Actions](https://george.mand.is/2020/02/publishing-bubos-rss-to-netlify-with-github-actions/)
## Getting Started
## Get Started
- Clone or fork the repo and run `npm install` to install the dependencies.
- Update `feeds.json` to include categories and links to feeds you would like to see.
@@ -18,19 +19,10 @@ You can read more about how this project came about on my blog:
That's it! You should now have a static page with links to the latest content from your feeds in the `public` folder, ready to serve.
## Differences in Bubo 2.0
Version 2.0 has introduced some substantial changes for Bubo! While the static output remains endearingly spartan, the engine that builds it has changed a bit.
Hopefully all of these changes are in services of making this project more useful to others and encouraging outside contributions.
Changes of note:
- Bubo has been rewritten in [TypeScript](https://www.typescriptlang.org/). It's pretty slick! I anticipate the typing could be improved, but it's a start.
- You fill find an `.nvmrc` file in the root of this project. Learn more [about nvm](https://github.com/nvm-sh/nvm) if you're unfamiliar.
- The script will actually write your `index.html` file for you (Previously the build script simply ran `node src/index.js > output/index.html`). It makes a strong assumption that this file lives in the `public` folder.
- There is a somewhat sophisticated mechansim in-place for batching & throttling your requests, if needed.
## Anatomy of Bubo Reader
<details>
<summary>
<strong>Anatomy of Bubo Reader</strong>
</summary>
The static pieces:
@@ -45,14 +37,19 @@ The engine:
- `src/renderer.ts` — The renderer that loads Nunjucks, the template and understands how to process the incoming feed data. Prefer something else? This is the place to change it!
- `src/utilities.ts` — A variety of parsing and normalization utilities for Bubo, hidden away to try and keep things clean.
## Throttling
</details>
In the main `index.ts` file you will find two values that allow yout to batch and throttle your feed requests:
<details>
<summary>
<strong>Throttling</strong>
</summary>
- `MAX_CONNECTIONS` dictates the maximium number of requests a batch can have going at once.
- `DELAY_MS` dictates the amount of de;ay time between each batch.
In the main `index.ts` file you will find two values that allow you to batch and throttle your feed requests:
The default configuration is **no batching or throttling** beacuse `MAX_CONNECTIONS` is set to `Infinity`. If you wanted to change Bubo to only fetch one feed at a time every second you could set these values to:
- `MAX_CONNECTIONS` dictates the maximum number of requests a batch can have going at once.
- `DELAY_MS` dictates the amount of delay time between each batch.
The default configuration is **no batching or throttling** because `MAX_CONNECTIONS` is set to `Infinity`. If you wanted to change Bubo to only fetch one feed at a time every second you could set these values to:
```javascript
const MAX_CONNECTIONS = 1;
@@ -67,22 +64,20 @@ const DELAY_MS = 2500;
```
In practice, I've never _really_ run into an issue leaving `MAX_CONNECTIONS` set to `Infinity` but this feels like a sensible safeguard to design.
## Demos
You can view live demos here:
</details>
- [https://bubo-rss-demo.netlify.com/](https://bubo-rss-demo.netlify.com/)
- [http://bubo-rss.glitch.me/](http://bubo-rss.glitch.me/)
Not the most exciting-looking demos, I'll admit, but they work!
**Getting Started**
<details>
<summary>
<strong>Getting Started</strong>
</summary>
- [Deploying to Glitch](#glitch)
- [Deploying to Netlify](#netlify)
- [Keeping feeds updated](#updated)
<a id="glitch"></a>
## Deploying to Glitch
The quickest way is to remix the project on Glitch:
@@ -93,6 +88,7 @@ There is also a `glitch` branch on this repo if you'd prefer to start there.
Just change some feeds in `./config/feeds.json` file and you're set! If you'd like to modify the style or the template you can changed `./public/style.css` file or the `./config/template.html` file respectively.
<a id="netlify"></a>
## Deploying to Netlify
- [Fork the repository](https://github.com/georgemandis/bubo-rss/fork)
@@ -102,6 +98,7 @@ Just change some feeds in `./config/feeds.json` file and you're set! If you'd li
The deploy settings should automatically import from the `netlify.toml` file. All you'll need to do is confirm and you're ready to go!
<a id="updated"></a>
### Keeping Feeds Updated
#### Using Netlify Webhooks
@@ -112,21 +109,29 @@ To keep your feeds up to date you'll want to [setup a Build Hook](https://www.ne
- [Zapier](https://zapier.com/)
- [EasyCron](https://www.easycron.com/)
#### Using GitHub Actions
Coming soon—there is an old branch that demonstrates this, but it needs to be revisisted in light of Bubo 2.0.
#### Rolling Your Own
If you already have a server running Linux and some command-line experience it might be simpler to setup a [cron job](https://en.wikipedia.org/wiki/Cron).
</details>
## Demos
You can view live demos here:
- [https://bubo-rss-demo.netlify.com/](https://bubo-rss-demo.netlify.com/)
- [http://bubo-rss.glitch.me/](http://bubo-rss.glitch.me/)
## Support
If you found this useful please consider sponsoring me or this project.
If you found this useful please consider [sponsoring me or this project](https://github.com/sponsors/georgemandis).
If you'd rather run this on your own server please consider using one of these affiliate links to setup a micro instance on [Linode](https://www.linode.com/?r=8729957ab02b50a695dcea12a5ca55570979d8b9), [Digital Ocean](https://m.do.co/c/31f58d367777) or [Vultr](https://www.vultr.com/?ref=8403978).
## Showcase
Here are some websites using Bubo Reader:
- [Kevin Fiol](https://kevinfiol.com/reader/) ([repo](https://github.com/kevinfiol/reader))
Please share if you would like to be featured!

View File

@@ -1,11 +1,12 @@
{
"Web Development": [
"Developer News": [
"https://hacks.mozilla.org/feed/",
"https://web.dev/feed.xml",
"https://v8.dev/blog.atom",
"https://alistapart.com/main/feed/",
"https://css-tricks.com/feed/",
"https://dev.to/feed"
"https://dev.to/feed",
"https://changelog.com/feed"
],
"Blogs": [
"https://george.mand.is/feed.xml",

1991
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{
"name": "bubo-reader",
"version": "2.0.1",
"version": "2.0.2",
"description": "A simple but effective feed reader (RSS, JSON)",
"homepage": "https://github.com/georgemandis/bubo-rss",
"main": "src/index.js",
@@ -27,19 +27,19 @@
},
"license": "MIT",
"dependencies": {
"chalk": "^5.0.0",
"node-fetch": "^3.1.0",
"nunjucks": "^3.2.0",
"rss-parser": "^3.6.3"
"chalk": "^5.2.0",
"node-fetch": "^3.3.1",
"nunjucks": "^3.2.4",
"rss-parser": "^3.13.0"
},
"devDependencies": {
"@types/node": "^16.11.10",
"@types/nunjucks": "^3.2.0",
"@types/xml2js": "^0.4.9",
"@typescript-eslint/eslint-plugin": "^5.4.0",
"@typescript-eslint/parser": "^5.4.0",
"eslint": "^8.3.0",
"tslib": "^2.3.1",
"typescript": "^4.5.2"
"@types/node": "^20.2.5",
"@types/nunjucks": "^3.2.2",
"@types/xml2js": "^0.4.11",
"@typescript-eslint/eslint-plugin": "^5.59.8",
"@typescript-eslint/parser": "^5.59.8",
"eslint": "^8.42.0",
"tslib": "^2.5.3",
"typescript": "^5.1.3"
}
}

View File

@@ -66,7 +66,11 @@ let completed = 0;
* and we want to build the static output.
*/
const finishBuild: () => void = async () => {
console.log("\nDone fetching everything!");
completed++;
// if this isn't the last feed, just return early
if (completed !== feedListLength) return;
process.stdout.write("\nDone fetching everything!\n");
// generate the static HTML output from our template renderer
const output = render({
@@ -77,10 +81,10 @@ const finishBuild: () => void = async () => {
// write the output to public/index.html
await writeFile("./public/index.html", output);
console.log(
process.stdout.write(
`\nFinished writing to output:\n- ${feedListLength} feeds in ${benchmark(
initTime
)}\n- ${errors.length} errors`
)}\n- ${errors.length} errors\n`
);
};
@@ -103,7 +107,6 @@ const processFeed =
}) =>
async (response: Response): Promise<void> => {
const body = await parseFeed(response);
completed++;
//skip to the next one if this didn't work out
if (!body) return;
@@ -124,20 +127,19 @@ const processFeed =
});
contentFromAllFeeds[group].push(contents as object);
console.log(
`${success("Successfully fetched:")} ${feed} - ${benchmark(startTime)}`
process.stdout.write(
`${success("Successfully fetched:")} ${feed} - ${benchmark(startTime)}\n`
);
} catch (err) {
console.log(
process.stdout.write(
`${error("Error processing:")} ${feed} - ${benchmark(
startTime
)}\n${err}`
)}\n${err}\n`
);
errors.push(`Error processing: ${feed}\n\t${err}`);
}
// if this is the last feed, go ahead and build the output
completed === feedListLength && finishBuild();
finishBuild();
};
// go through each group of feeds and process
@@ -150,15 +152,16 @@ const processFeeds = () => {
for (const feed of feeds) {
const startTime = Date.now();
setTimeout(() => {
console.log(`Fetching: ${feed}...`);
process.stdout.write(`Fetching: ${feed}...\n`);
fetch(feed)
.then(processFeed({ group, feed, startTime }))
.catch(err => {
console.log(
error(`Error fetching ${feed} ${benchmark(startTime)}`)
process.stdout.write(
error(`Error fetching ${feed} ${benchmark(startTime)}\n`)
);
errors.push(`Error fetching ${feed} ${err.toString()}`);
errors.push(`Error fetching ${feed} ${err.toString()}\n`);
finishBuild();
});
}, (idx % (feedListLength / MAX_CONNECTIONS)) * DELAY_MS);
idx++;