Continued:

- throw exception again, if they happen then they won't be fixed within a
  split of a second
- also make sure that home directory of FBA is properly set, sure you can
  choose a different directory or take the default /home/fba/
- added recheck.sh, a small wrapper script I wrote for myself and you should
  try. For example above exceptions might cause the used software not being
  detected (sure with timeouts) then you can run ./recheck.sh --software
  to re-test them

diff --git a/README.md b/README.md
deleted file mode 100644 (file)
index db37a0e..0000000
--- a/README.md
+++ /dev/null
@@ -1,71 +0,0 @@
-# Fedi block API
-
-Used to see which instances block yours.
-
-## software used:
-
-- python 3.10.2
-
-## Installation
-
-```bash
-sudo useradd -m fba
-sudo mkdir -p /opt/fedi-block-api
-sudo chown -R fba:fba /opt/fedi-block-api
-sudo -Hu fba git clone git://git.mxchange.org/fba.git /opt/fedi-block-api/
-cd /opt/fedi-block-api/
-sudo -Hu fba pip3 install -r requirements.txt
-sudo -Hu fba cp blocks_empty.db blocks.db
-sudo -Hu fba cp config.defaults.json config.json
-sudo -Hu fba ./fba.py fetch_instances --domain=mastodon.social # try a bunch of large servers here
-```
-
-### Alter configuration file
-You maybe wish to change the configuration file, e.g. log_level is set to "info" which is the default but invates privacy of your users, but your choice:
-
-```
-    "critical"
-    "error"
-    "warning"
-    "info"
-    "debug"
-    "trace"
-```
-
-### Fetch blocklists locally (WIP)
-To save yourself bandwidth and codeberg, too, you want to clone the blocklists from some folks:
-
-```
-sudo -Hu fba git clone https://codeberg.org/oliphant/blocklists.git blocklists/oliphant/
-```
-
-WIP notice: This feature is not implemented yet.
-
-### Install the services
-
-```bash
-sudo cp services/* /etc/systemd/system
-```
-
-### start the services
-
-```bash
-systemctl enable --now fetch_blocks
-systemctl enable --now fedi_block_api
-```
-
-## Try it out
-
-https://fba.ryona.agency/ uses an older code than this one. So it doesn't reflect this code here. My FBA sub-domain is currently password-protected as I still need to fix some performance issues with large blocking lists.
-
-## Maintenance
-
-Run these SQL queries from time to time. They always should return zero.
-
-```sql
-SELECT COUNT(blocked) AS cnt FROM blocks LEFT JOIN instances ON blocks.blocked = instances.domain OR blocks.blocker = instances.domain WHERE domain IS NULL LIMIT 1;
-SELECT COUNT(domain) AS cnt FROM instances WHERE nodeinfo_url IS NOT NULL AND software IS NULL LIMIT 1;
-```
-## License
-
-[AGPLv3](https://gnu.org)

diff --git a/docs/README.md b/docs/README.md
new file mode 100644 (file)
index 0000000..8c08479
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,71 @@
+# Fedi block API
+
+Used to see which instances block yours.
+
+## software used:
+
+- python 3.10.2
+
+## Installation
+
+```bash
+sudo useradd --home-dir=/opt/fedi-block-api --comment "Fedi Block API" --user-group --create-home fba
+sudo mkdir -p /opt/fedi-block-api
+sudo chown -R fba:fba /opt/fedi-block-api
+sudo -Hu fba git clone git://git.mxchange.org/fba.git /opt/fedi-block-api/
+cd /opt/fedi-block-api/
+sudo -Hu fba pip3 install -r requirements.txt
+sudo -Hu fba cp blocks_empty.db blocks.db
+sudo -Hu fba cp config.defaults.json config.json
+sudo -Hu fba ./fba.py fetch_instances --domain=mastodon.social # try a bunch of large servers here
+```
+
+### Alter configuration file
+You maybe wish to change the configuration file, e.g. log_level is set to "info" which is the default but invates privacy of your users, but your choice:
+
+```
+    "critical"
+    "error"
+    "warning"
+    "info"
+    "debug"
+    "trace"
+```
+
+### Fetch blocklists locally (WIP)
+To save yourself bandwidth and codeberg, too, you want to clone the blocklists from some folks:
+
+```
+sudo -Hu fba git clone https://codeberg.org/oliphant/blocklists.git blocklists/oliphant/
+```
+
+WIP notice: This feature is not implemented yet.
+
+### Install the services
+
+```bash
+sudo cp services/* /etc/systemd/system
+```
+
+### start the services
+
+```bash
+systemctl enable --now fetch_blocks
+systemctl enable --now fedi_block_api
+```
+
+## Try it out
+
+https://fba.ryona.agency/ uses an older code than this one. So it doesn't reflect this code here. My FBA sub-domain is currently password-protected as I still need to fix some performance issues with large blocking lists.
+
+## Maintenance
+
+Run these SQL queries from time to time. They always should return zero.
+
+```sql
+SELECT COUNT(blocked) AS cnt FROM blocks LEFT JOIN instances ON blocks.blocked = instances.domain OR blocks.blocker = instances.domain WHERE domain IS NULL LIMIT 1;
+SELECT COUNT(domain) AS cnt FROM instances WHERE nodeinfo_url IS NOT NULL AND software IS NULL LIMIT 1;
+```
+## License
+
+[AGPLv3](https://gnu.org)

diff --git a/fba/http/nodeinfo.py b/fba/http/nodeinfo.py
index 7d7620ad9d658240216011bb2108dbe5f76fc0da..ec78862ef6fac91fa9388189d5712d1f671cae3d 100644 (file)
--- a/fba/http/nodeinfo.py
+++ b/fba/http/nodeinfo.py
@@ -42,7 +42,10 @@ def fetch_nodeinfo(domain: str, path: str = None) -> dict:
     data = fetch_wellknown_nodeinfo(domain)
 
     logger.debug("data[%s](%d)='%s'", type(data), len(data), data)
-    if "error_message" not in data and "json" in data and len(data["json"]) > 0:
+    if "exception" in data:
+        logger.warning("Exception returned: '%s', raising again ...", type(data["exception"]))
+        raise data["exception"]
+    elif "error_message" not in data and "json" in data and len(data["json"]) > 0:
         logger.debug("Invoking instances.set_last_nodeinfo(%s) ...", domain)
         instances.set_last_nodeinfo(domain)
 

diff --git a/recheck.sh b/recheck.sh
new file mode 100755 (executable)
index 0000000..e685ca7
--- /dev/null
+++ b/recheck.sh
@@ -0,0 +1,51 @@
+#!/bin/sh
+
+MODE=""
+if [ "$1" = "--help" ]
+then
+       echo "Usage: $ [file|--software|--nodeinfo|--detection|--no-auto]"
+       exit 255
+elif [ -n "$1" -a -f "$1" ]
+then
+       DOMAINS=$(cat "$1")
+       MODE="file"
+elif [ "$1" = "--software" ]
+then
+       DOMAINS=$(sqlite3 blocks.db "SELECT domain FROM instances WHERE software IS NULL ORDER BY last_updated ASC;")
+       MODE="software"
+elif [ "$1" = "--nodeinfo" ]
+then
+       DOMAINS=$(sqlite3 blocks.db "SELECT domain FROM instances WHERE software IS NULL AND nodeinfo_url IS NOT NULL ORDER BY last_updated ASC;")
+       MODE="nodeinfo"
+elif [ "$1" = "--detection" ]
+then
+       DOMAINS=$(sqlite3 blocks.db "SELECT domain FROM instances WHERE detection_mode IS NULL ORDER BY last_updated ASC;")
+       MODE="detection"
+elif [ "$1" = "--no-auto" ]
+then
+       DOMAINS=$(sqlite3 blocks.db "SELECT domain FROM instances WHERE detection_mode != 'AUTO_DISCOVERY' ORDER BY last_updated ASC;")
+       MODE="noauto"
+else
+       DOMAINS=$(sqlite3 blocks.db "SELECT domain FROM instances WHERE software IS NULL AND nodeinfo_url IS NOT NULL ORDER BY last_updated ASC;")
+fi
+
+if [ -z "${DOMAINS}" ]
+then
+       echo "$0: No domains found! MODE='${MODE}'"
+       exit 255
+fi
+
+for DOMAIN in ${DOMAINS};
+do
+       echo "$0: DOMAIN='${DOMAIN}'"
+       if [ -n "$1" -a -z "${MODE}" ]
+       then
+               ./fba.py update_nodeinfo --domain=${DOMAIN} "$1"
+       elif [ -n "$2" ]
+       then
+               ./fba.py update_nodeinfo --domain=${DOMAIN} "$2"
+       else
+               ./fba.py update_nodeinfo --domain=${DOMAIN}
+       fi
+done
+echo "$0: All done."