]> git.mxchange.org Git - fba.git/log
projects / fba.git / log
? search:
summary | shortlog | log | commit | commitdiff | tree
first ⋅ prev ⋅ next
fba.git
4 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 23 Jun 2024 11:28:52 +0000 (13:28 +0200)]
Continued:
- 'neodb' might appear in names such as 'testando neodb'

5 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 15 Jun 2024 13:11:40 +0000 (15:11 +0200)]
Continued:
- if the found digest isn't valid, then set it to None
- ignore (not skip) any invalid digests

5 months agoContinued:
commit | commitdiff | tree
Roland Häder [Fri, 14 Jun 2024 16:48:25 +0000 (18:48 +0200)]
Continued:
- fixed exception, need to check is_registered() here first

5 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 2 Jun 2024 14:53:24 +0000 (16:53 +0200)]
Continued:
- show run-mode of nodeinfo.sh
- fetch_instances can now (again) fetch new instances (not registered)

5 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 1 Jun 2024 19:21:39 +0000 (21:21 +0200)]
Continued:
- added another UTF-8 exception

5 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 25 May 2024 16:55:09 +0000 (18:55 +0200)]
Continued:
- added another exception

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 20 May 2024 19:23:31 +0000 (21:23 +0200)]
Continued:
- Hugo isn't distributed at all. I guess someone replaced a federated software
  with Hugo at the same domain

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 20 May 2024 14:36:51 +0000 (16:36 +0200)]
Continued:
- added 'biblioreads' as peer list provider

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 20 May 2024 12:49:13 +0000 (14:49 +0200)]
Continued:
- variable `response_url` is the whole URL, maybe stripped off from its anchor
  (#) part but it is NEVER a valid domain (ops)

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 20 May 2024 12:33:25 +0000 (14:33 +0200)]
Continued:
- some servers may return '<html something>' at the start (position 0),
- so if string.find() cannot find that string "<html", it will return -1 instead
  of zero

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 11 May 2024 19:36:59 +0000 (21:36 +0200)]
Continued:
- dict element 'domain' will always be there, no double-redundant checks

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 11 May 2024 19:30:26 +0000 (21:30 +0200)]
Continued:
- fetch_blocks_from_about() does return a 2-dimensional array, not a flat one

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Fri, 10 May 2024 20:56:07 +0000 (22:56 +0200)]
Continued:
- network aka. "universe" 'snac' added as peer source

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 4 May 2024 00:05:29 +0000 (02:05 +0200)]
Continued:
- addedd alias 'cutiekey' for 'misskey'

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Thu, 2 May 2024 01:23:12 +0000 (03:23 +0200)]
Continued:
- serveo.net blacklisted: 'Expose local servers to the internet'

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 30 Apr 2024 13:12:54 +0000 (15:12 +0200)]
Continued:
- nodeinfo_url should never be None/NULL here

6 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 23 Apr 2024 23:46:57 +0000 (01:46 +0200)]
Continued:
- blocked 'documentation.on.seirdy.one'
- removed old-lost blocklists
- added another CSV file for seirdy.one

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 21 Apr 2024 17:49:16 +0000 (19:49 +0200)]
Continued:
- rewrite all "*mammuthus*" to "mammuthus"

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Fri, 19 Apr 2024 23:41:39 +0000 (01:41 +0200)]
Continued:
- recheck_obfuscation now supports deleting unwanted domains from both tables
  'instances' and 'blocks'

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Fri, 19 Apr 2024 15:41:20 +0000 (17:41 +0200)]
Continued:
- moved some pre-sorting

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Thu, 18 Apr 2024 17:23:45 +0000 (19:23 +0200)]
Continued:
- added alias 'trashposs' for 'misskey'

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 17 Apr 2024 20:50:46 +0000 (22:50 +0200)]
Continued:
- added uncached blocks.get_reason()
- update block reason when current isn't set but updated reason is set

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 17 Apr 2024 19:38:13 +0000 (21:38 +0200)]
Continued:
- renamed local (overwritten) variable 'blocked' to 'cleared' so the array
  index can be found again

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 17 Apr 2024 18:10:06 +0000 (20:10 +0200)]
Continued:
- need to check if variable 'blocked' is None or an empty string first
- log size (len()) not whole array

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 16 Apr 2024 22:26:14 +0000 (00:26 +0200)]
Continued:
- added some missing instances.is_recent(foo) invocations
- use last_instance_fetch (default) where ever federation.fetch_instances() is
  been invoked
- wording fixed

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 16 Apr 2024 22:08:39 +0000 (00:08 +0200)]
Continued:
- added alias 'tanukey' for 'misskey'

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 16 Apr 2024 14:23:50 +0000 (16:23 +0200)]
Continued:
- blacklisted 'clients.your-server.de' and 2 others as this is a reverse-IP host
  name, but instances should have proper domain names

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 14 Apr 2024 09:54:15 +0000 (11:54 +0200)]
Continued:
- fixed 'TypeError: not enough arguments for format string'

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 10 Apr 2024 21:53:04 +0000 (23:53 +0200)]
Continued:
- cache blacklist function invocations

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 10 Apr 2024 21:38:28 +0000 (23:38 +0200)]
Continued:
- added 'toki' for peer lists, you can find it here:
  https://github.com/purifetchi/Toki

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 10 Apr 2024 09:56:11 +0000 (11:56 +0200)]
Continued:
- python3-aiofiles needs to be installed manually with Python 3.9
- Python 3.11 seem to install it automatically

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 9 Apr 2024 16:11:37 +0000 (18:11 +0200)]
Continued:
- added missing keyword (ops)

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 2 Apr 2024 01:39:32 +0000 (03:39 +0200)]
Continued:
- removed entries as the list has been deleted by oliphant + they provide
  their own version already

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 1 Apr 2024 22:56:35 +0000 (00:56 +0200)]
Continued:
- FBA now requires Python 3.11
- updated documentation accordingly
- removed fastapi and uvicorn requirements, your distribution must now provide
  it
- removed jinja2 and requests
- ignore entire venv/ directory, contains local paths

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 1 Apr 2024 03:12:17 +0000 (05:12 +0200)]
Continued:
- updated README file to reflect virtual environment for Python 3.11
- README file is a bit better

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 1 Apr 2024 03:01:19 +0000 (05:01 +0200)]
Continued:
- switched from global path to virtual-environment path

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 1 Apr 2024 02:49:23 +0000 (04:49 +0200)]
Continued:
- Python 3.11 needs a virtual environment
- so please add

source "${HOME}/fba/venv/bin/activate"

... to your ~/.bashrc

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 31 Mar 2024 21:05:47 +0000 (23:05 +0200)]
Continued:
- reduced duplicated code into a local variable 'response_url'
- need to recheck validity of 'response_url'

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 31 Mar 2024 08:00:43 +0000 (10:00 +0200)]
Continued:
- logged types (string or 'None' type)

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 31 Mar 2024 05:01:10 +0000 (07:01 +0200)]
Continued:
- handled ParserRejectedMarkup properly

7 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 31 Mar 2024 01:09:40 +0000 (03:09 +0200)]
Continued:
- added alias 'nexkey' for 'misskey'

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 20 Mar 2024 17:09:34 +0000 (18:09 +0100)]
Continued:
- added 'piefed' as possible source for block lists, still the network lacks
  API for instance list retrival

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 17 Mar 2024 00:07:51 +0000 (01:07 +0100)]
Continued:
- that Wiki page was listing "evil instances" and now explains the view of the
  owner of what FediBlock is

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 12 Mar 2024 12:50:39 +0000 (13:50 +0100)]
Continued:
- added 'hugo' as a source for more peers (only one so far)

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 12 Mar 2024 10:27:49 +0000 (11:27 +0100)]
Continued:
- added new aliases for pleroma
- moved pleroma aliases to "private" module variable

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 11 Mar 2024 21:28:00 +0000 (22:28 +0100)]
Continued:
- added aliases for misskey, mastodon and pleroma (each one)
- moved mastodon aliases to module ("private") variable, please don't access it
  outside the module!

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 11 Mar 2024 09:18:12 +0000 (10:18 +0100)]
Continued:
- fixed SQL string
- partly reverted because simple code like 'tuple(row)' doesn't work

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Mar 2024 10:57:12 +0000 (11:57 +0100)]
Continued:
- need to convert type Row to a true tuple

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Mar 2024 10:53:17 +0000 (11:53 +0100)]
Continued:
- row[] does already contain named keys, e.g. 'total_websites' so let's alias
  the COUNT() statement to such key name which reduces some code
- renamed 'blocks_recorded' to 'total_blocks'

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Mar 2024 10:35:19 +0000 (11:35 +0100)]
Continued:
- renamed known_instances to total_websites because this is more clear
- you cannot distinguish between a regular website and a former Fediverse
  instance (e.g. before: Mastodon was installed, now IBM Connections)

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 9 Mar 2024 20:16:10 +0000 (21:16 +0100)]
Continued:
- check 'json' key

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Fri, 8 Mar 2024 15:43:10 +0000 (16:43 +0100)]
Continued:
- added missing key 'json' (ops!)

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 3 Mar 2024 03:37:26 +0000 (04:37 +0100)]
Continued:
- added 7988276.xyz as a testing/development hoster

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 3 Mar 2024 03:25:11 +0000 (04:25 +0100)]
Continued:
- added another alias "hijikey" for "misskey"

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Fri, 1 Mar 2024 06:36:44 +0000 (07:36 +0100)]
Continued:
- moved --same from nodeinfo.sh to command update_nodeinfo()

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 28 Feb 2024 17:12:22 +0000 (18:12 +0100)]
Continued:
- also log parameter 'column'
- blocks.add() should not be invoked with 'blocker' is already blocking
  'blocked' at 'block_level'

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 25 Feb 2024 02:43:29 +0000 (03:43 +0100)]
Continued:
- added another gardenfence blocklist for bootstrapping FBA
- just run ./fba.py fetch_txt and you can build up an initial list of peers

8 months agoContinued:
commit | commitdiff | tree
Roland Häder [Thu, 22 Feb 2024 23:38:57 +0000 (00:38 +0100)]
Continued:
- more pre-checks to avoid exceptions
- don't abuse catching them here as a control statement (if)

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Thu, 15 Feb 2024 17:58:51 +0000 (18:58 +0100)]
Continued:
- added network type 'vebinet' (Mozilla's Fediverse software) for peer retrival
- domain blocks aren't supported (yet)

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Thu, 15 Feb 2024 13:30:07 +0000 (14:30 +0100)]
Continued:
- first simple variable checks
- next "lighter" functions being invoked
- last "heavy" function, which means possible database queries

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 13 Feb 2024 18:31:31 +0000 (19:31 +0100)]
Continued:
- added some more plain-text block lists (thanks to @Kromonos)

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 12 Feb 2024 15:45:46 +0000 (16:45 +0100)]
Continued:
- check parameters first, then "expensive" function invocation's returned value
- command fetch_instances should always fetch instances from a provided domain
  name
- only when --software=bar is used and no --force parameter was given

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Thu, 8 Feb 2024 17:00:57 +0000 (18:00 +0100)]
Continued:
- make --domain parameter for command fetch_instances optional so --software
  can be handled, too
- skip recently crawled domains in same loop
- args.<domain|software> can be both of type None, too

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 7 Feb 2024 19:26:24 +0000 (20:26 +0100)]
Continued:
- github isn't any fediverse instance
- https://github.io redirects to a bad URL

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 7 Feb 2024 07:55:13 +0000 (08:55 +0100)]
Continued:
- it was imported from previous code and seem not to work everywhere
- so let's remove the non-working part

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 4 Feb 2024 08:29:38 +0000 (09:29 +0100)]
Continued:
- added alias 'rosekey' for 'misskey'
- also 'smithereen' is a federating (at least peer list is provided) software

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 3 Feb 2024 19:39:39 +0000 (20:39 +0100)]
Continued:
- maybe there is no unsorted list or table at all

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 24 Jan 2024 17:14:14 +0000 (18:14 +0100)]
Continued:
- 'continue' if an exception is thrown because that instance can be "ignored"

9 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 24 Jan 2024 05:18:53 +0000 (06:18 +0100)]
Continued:
- handled unregistered instances

10 months agoContinued/WIP:
commit | commitdiff | tree
Roland Häder [Tue, 16 Jan 2024 19:18:16 +0000 (20:18 +0100)]
Continued/WIP:
- in commands.fetch_instances() added initialization of variables 'rows'
- also moved fetching rows into if() block
- commented some more code

10 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 16 Jan 2024 02:48:35 +0000 (03:48 +0100)]
Continued:
- added alias "kmyblue" for mastodon

10 months agoWIP:
commit | commitdiff | tree
Roland Häder [Sun, 14 Jan 2024 01:25:10 +0000 (02:25 +0100)]
WIP:
- rewritten fetch_instances towars allowing also --software=foo as an
  alternative parameter

10 months agoContinued:
commit | commitdiff | tree
Roland Häder [Fri, 12 Jan 2024 09:00:40 +0000 (10:00 +0100)]
Continued:
- changed string "true/false" to boolean
- please update your configuration file (all occurances!)

10 months agoContinued:
commit | commitdiff | tree
Roland Häder [Fri, 12 Jan 2024 03:56:07 +0000 (04:56 +0100)]
Continued:
- need to negate this state: only skip/reject .i2p domains when they are not
  allowed by configuration (which is default)

10 months agoContinued:
commit | commitdiff | tree
Roland Häder [Wed, 10 Jan 2024 21:52:50 +0000 (22:52 +0100)]
Continued:
- moved utils.fetch_url() to module network as this is network-related
- added some debug lines

10 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 2 Jan 2024 19:45:51 +0000 (20:45 +0100)]
Continued:
- need to remove prefix "re:" before cleaning up software, else all software is called "re"
- added alias "lovers" for "misskey"

10 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 2 Jan 2024 19:30:01 +0000 (20:30 +0100)]
Continued:
- added alias "miraiskey" for "misskey"
- moved all those aliases to "private" variable _misskey_aliases

10 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 23 Dec 2023 16:10:59 +0000 (17:10 +0100)]
Continued:
- fediverse.observer has changed their API to Graph (JSON POST)
- domain singleuser.club blacklisted, this domain started flooding with
  sub-domains which have wwXX as another sub-domain

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Fri, 22 Dec 2023 07:44:52 +0000 (08:44 +0100)]
Continued:
- need to cut off everything after hash symbol because that is for JavaScript
  click-event loaded content anyway
- prevented a few empty/None strings for invoking tidyup.domain()
- improved a few log messages

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 16 Dec 2023 13:19:57 +0000 (14:19 +0100)]
Continued:
- don't allow empty peer list in federation.add_peers()

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 16 Dec 2023 08:51:12 +0000 (09:51 +0100)]
Continued:
- added missing 'continue'
- added some debug messages
- skip empty/NoneType domain names

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 12 Dec 2023 06:07:23 +0000 (07:07 +0100)]
Continued:
- maybe error[json][error] is not a dict

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Tue, 12 Dec 2023 02:34:16 +0000 (03:34 +0100)]
Continued:
- added "error_message" JSON error message

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Mon, 11 Dec 2023 06:57:39 +0000 (07:57 +0100)]
Continued:
- also check blacklist first, before invoking instances.is_recent()

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 23:54:01 +0000 (00:54 +0100)]
Continued:
- also check blacklist in instances.is_recent() function
- first check against blacklist, then if it is recent (in commands.py)

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 22:44:35 +0000 (23:44 +0100)]
Continued:
- added alias 'goblin' for 'misskey'
- added debug line

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 22:08:50 +0000 (23:08 +0100)]
Continued:
- Python still needs a backslash here

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 08:04:23 +0000 (09:04 +0100)]
Continued:
- check validity of href URL before parsing it (controlled skip instead of
  uncontrolled raised exception)

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 07:20:07 +0000 (08:20 +0100)]
Continued:
- reformatted

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 05:15:09 +0000 (06:15 +0100)]
Continued:
- some relays may have removed/never set nodeinfo URL, these need to be skipped
  to avoid exception in fetch_url() function

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 04:50:06 +0000 (05:50 +0100)]
Continued:
- command 'fetch_txt' supports now --force parameter
- check against blacklist (wasn't in is_registered())
- some debug messages added

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 04:06:26 +0000 (05:06 +0100)]
Continued:
- formatted SQL statements

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 02:42:22 +0000 (03:42 +0100)]
Continued:
- forgot to vacuum-clean it

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sun, 10 Dec 2023 01:06:59 +0000 (02:06 +0100)]
Continued:
- simplified code, no need for this extra indenting

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 9 Dec 2023 07:15:42 +0000 (08:15 +0100)]
Continued:
- combined None and "" (empty string) for lesser code
- added/improved debug messages
- got rid of one local variable, added another one ;-)

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 9 Dec 2023 07:03:01 +0000 (08:03 +0100)]
Continued:
- improved logger messages

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 9 Dec 2023 05:14:48 +0000 (06:14 +0100)]
Continued:
- logged domain name
- better check variable software against None and empty string

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 9 Dec 2023 05:04:20 +0000 (06:04 +0100)]
Continued:
- initially set original_software, maybe it needs fixing later

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 9 Dec 2023 05:01:38 +0000 (06:01 +0100)]
Continued:
- need to catch any network-related exceptions to reduce _DEPTH counter

11 months agoContinued:
commit | commitdiff | tree
Roland Häder [Sat, 9 Dec 2023 04:10:38 +0000 (05:10 +0100)]
Continued:
- moved original_software next to 'software' column
- replaced own _cache dict with @lru_cache

next
Personal clone of "fedi-block-api", if you like to use the command fetch_instances_social you need to get an API key from https://instances.social .