89 lines
2.5 KiB
Markdown
89 lines
2.5 KiB
Markdown
# Beaumontmeteo
|
|
|
|
Shiny app and helper scripts to cache Meteo France station data around Beaumont in a local SQLite database.
|
|
|
|
## What is stored
|
|
|
|
The cache now uses one unified dataset called `weather_measurements` inside `data/rain_history.sqlite`.
|
|
|
|
Each row stores:
|
|
|
|
- location,
|
|
- station,
|
|
- observation timestamp,
|
|
- metric id,
|
|
- numeric value,
|
|
- source metadata.
|
|
|
|
That means rain, temperature, humidity, wind and pressure can all be compared on the same time axis later.
|
|
|
|
## App
|
|
|
|
Run from the project root:
|
|
|
|
```r
|
|
shiny::runApp()
|
|
```
|
|
|
|
The UI now has two panels:
|
|
|
|
- a rain panel backed by historical `DPClim` rainfall,
|
|
- a second panel backed by the same SQLite dataset, with a selector for temperature and other station metrics.
|
|
|
|
## Syncing the cache
|
|
|
|
Put your Meteo France API key in `data/secrets` as `token4`, then run:
|
|
|
|
```bash
|
|
Rscript scripts/update_rain_db.R
|
|
```
|
|
|
|
For a slow historical rainfall backfill into the same SQLite dataset:
|
|
|
|
```bash
|
|
Rscript scripts/backfill_two_years.R
|
|
```
|
|
|
|
Useful options:
|
|
|
|
- `update_rain_db.R --location=vignasses`
|
|
- `update_rain_db.R --location=vignasses,la_mure`
|
|
- `update_rain_db.R --days=60`
|
|
- `update_rain_db.R --db-path=/somewhere/else/rain.sqlite`
|
|
- `backfill_two_years.R --location=vignasses`
|
|
- `backfill_two_years.R --db-path=/somewhere/else/rain.sqlite`
|
|
- `backfill_two_years.R --chunk-days=30`
|
|
- `backfill_two_years.R --resume=false`
|
|
- `backfill_two_years.R --between-chunk-sleep=20`
|
|
|
|
## Important note about frequency
|
|
|
|
Historical rain can be backfilled from the climatology endpoint, so a weekly sync is fine there.
|
|
|
|
Temperature, humidity, wind and pressure come from the rolling station observation feed. To keep those metrics continuous in the database, run the sync daily rather than weekly.
|
|
|
|
## Cron examples
|
|
|
|
Daily full weather sync:
|
|
|
|
```bash
|
|
0 7 * * * cd /path/to/beaumontmeteo && Rscript scripts/update_rain_db.R >> /tmp/beaumontmeteo-sync.log 2>&1
|
|
```
|
|
|
|
If you only care about rain history, the same script can still be run weekly because rainfall is backfilled on each sync.
|
|
|
|
## Why SQLite
|
|
|
|
SQLite is a good fit here because this is a local, read-heavy app with a small periodic write job.
|
|
|
|
- No database server to install or operate.
|
|
- One file to move, back up and inspect.
|
|
- Proper indexed queries and deduplication, unlike CSV or RDS files.
|
|
- The app still works offline once data has been cached.
|
|
|
|
If this ever becomes a shared multi-user service with concurrent writers, Postgres would be the next upgrade.
|
|
|
|
## Notes
|
|
|
|
- The database path can be overridden with `BEAUMONT_RAIN_DB`.
|
|
- The repo ignores `data/*.sqlite` so the cache file stays local.
|