Refine info on downloading articles

This commit is contained in:
2021-01-06 01:15:17 -08:00
parent 21cfc1b157
commit f80a820a19
+23 -4
View File
@@ -10,7 +10,7 @@ In the root of the project, do the following.
# Install node modules
npm i
# Initialize database
# Initialize database from scratch
sqlite3 main.db < sql/ds.sql
sqlite3 main.db < sql/ds.001.sql
sqlite3 main.db < sql/ds.002.sql
@@ -19,9 +19,28 @@ sqlite3 main.db < sql/ds.002.sql
wget http://dss.coincidencedetector.com/archive/dss-latest.db.gz
gzcat dss-latest.db.gz > main.db
# Scrape articles; This may take a few days if done from scratch.
bin/download-all
# Run the server
micro-dev -p 1984 site.js
```
## Downloading Articles
```sh
# Download all articles from scratch:
# This takes a few days and may need babysitting in case of crashes.
bin/download-all
# Dowloading the last 140 articles per category:
# If you have a recent snapshot, running this once is how you should get it up to date.
P=1 bin/download-all
```
## Cron
To update the database with new articles, replace the value of `DSS_ROOT` with the path
to where DSS is on your system, and place the following in your crontab.
```sh
DSS_ROOT=/path/to/dss
55 * * * * cd $DSS_ROOT && bin/articles-update
```