~erin
|
ed53ec320e
|
Fix panic, preventing full crawling
|
2023-07-25 20:11:29 -04:00 |
~erin
|
c141135847
|
Sanitize HTML?
|
2023-07-25 19:54:05 -04:00 |
~erin
|
758c16b78b
|
Justfile
|
2023-07-25 19:47:39 -04:00 |
~erin
|
561ef2dfb4
|
Test database
|
2023-07-25 19:28:17 -04:00 |
~erin
|
541deca5f6
|
Add README
|
2023-07-25 19:27:56 -04:00 |
~erin
|
326a6b8042
|
Should crawl pages after a certain age
|
2023-07-25 18:33:38 -04:00 |
~erin
|
57684c037e
|
Fix recursive crawling
|
2023-07-25 18:16:45 -04:00 |
~erin
|
159164674e
|
Crawl through URLs via allowlist
|
2023-07-25 17:36:41 -04:00 |
~erin
|
892dc35f74
|
Crawl all at once, test cacache
|
2023-07-25 16:58:25 -04:00 |
~erin
|
86f6d1a631
|
Fetch pages in sitemap
|
2023-07-25 16:11:09 -04:00 |
~erin
|
bc062a39f3
|
Use basic configuration file
|
2023-07-25 15:24:14 -04:00 |
~erin
|
03f3cfd5c0
|
config changes
|
2023-07-25 14:59:47 -04:00 |
~erin
|
971b217a94
|
Basic project setup
|
2023-07-25 14:52:19 -04:00 |