Roland Häder [Fri, 9 Jun 2023 08:21:38 +0000 (10:21 +0200)]
Continued:
- ignored more files (tracing, coverage)
- rewrote towards fba.csrf.determine() so it will always be determined, causes
another request unfortunately
- or can we somehow save the cookie jar ?
- logging of errors can now be disabled for debug purposes
Roland Häder [Fri, 9 Jun 2023 03:32:23 +0000 (05:32 +0200)]
Continued:
- the repository is more up-to-date, let's fetch this instead
- had to parse the markdown code back to HTML to be able to traverse with
existing code over it
Roland Häder [Fri, 9 Jun 2023 00:50:37 +0000 (02:50 +0200)]
Fixed some issues found by pylint:
- added pylint.rc file (all checks are enabled)
- '!= None' should be 'is not None'
- '== None' should be 'is None'
- '!= type' should be 'not isinstance(var, type)'
- fixed some 'unused variable'
- fixed 'duplicate definition'
- fixed 'invalid name'
- fixed f"foo" with no {var} in it
- renamed more variables
Roland Häder [Thu, 8 Jun 2023 23:58:53 +0000 (01:58 +0200)]
Continued:
- added command 'fetch_federater' which fetches a CSV file from github
- if software is null, attempt to determine it
- renamed get_url() to fetch_url()
Roland Häder [Thu, 8 Jun 2023 18:03:43 +0000 (20:03 +0200)]
Continued:
- some block reasons have a long domain list (which can be taken as
instance names, too) with no space after a comma
- this causes the web page to be very(!) wide
- added missing "import inspect"
- improved some debug messages
Roland Häder [Thu, 8 Jun 2023 16:50:52 +0000 (18:50 +0200)]
Continued:
- also tidyup reason after fetching from instance
- moved validating domain to upper if/elif blocks
- re-color website to more friendlier colors
- rewrote some debug lines
Roland Häder [Thu, 8 Jun 2023 14:43:04 +0000 (16:43 +0200)]
Continued:
- moved templates to templates/views/
- introduced base.html which should be extended by view templates
- TOP x lists now have an own view template
Roland Häder [Tue, 6 Jun 2023 18:43:41 +0000 (20:43 +0200)]
WIP:
- moved network-specific function to their own fba/network/<network>.py module
- renamed blocker -> domain in network-specific functions (it is 'domain' in
the end)
Roland Häder [Tue, 6 Jun 2023 10:16:56 +0000 (12:16 +0200)]
Continued:
- renamed result <-> blocks
- added config entry 'hostname' which you should change to where you have
installed it, e.g. mine is 'fba.mxchange.org' (password-protected)
Roland Häder [Tue, 6 Jun 2023 09:44:56 +0000 (11:44 +0200)]
Continued:
- added more for <title> so search engines have more pages to index and not
combine them into one result only
- still we need paging support here
Roland Häder [Tue, 6 Jun 2023 09:35:11 +0000 (11:35 +0200)]
Continued:
- `inspect.currentframe().f_code.co_name` is one LARGE way for finding out
current function's name but it does not involve any private functions
being called publicly
- still PHP has __FUNCTION__ for this purpose ...
Roland Häder [Mon, 5 Jun 2023 23:18:13 +0000 (01:18 +0200)]
Continued:
- introduced get_url() which uses urllib.parse() to wrap invocation around
get_response() nicely
- so all reqto.get|post() invocations are now centralized for later easy
expansion with SOCKS proxy support
Roland Häder [Mon, 5 Jun 2023 21:41:07 +0000 (23:41 +0200)]
Continued:
- add argument --single to commands.fetch_instances() which only fetches given
instance
- int() doesn't "magically" turn NoneType to 0, PHP would do it, so let's handle
this here by ourself
Roland Häder [Mon, 5 Jun 2023 20:09:42 +0000 (22:09 +0200)]
Continued:
- introduced argparse which is a more flexible way of handling command-line
arguments
- moved all commands to fba/command.py you can now access them through
$ ./fba.py <command>
- please use --help to see which commands are all supported, you can also use
it on a single command to get all supported arguments
Roland Häder [Mon, 5 Jun 2023 07:31:18 +0000 (09:31 +0200)]
Continued:
- some APIs return the same result at the end all over again
- so we need to check if the domain is already part of the created list
- for that purpose has_element() is introduced
Roland Häder [Sun, 4 Jun 2023 12:54:47 +0000 (14:54 +0200)]
Continued:
- introduced json_from_response() which handles decoding errors, e.g. when
a server has returned a HTML instead of a JSON which is caused by improper
error handling
Roland Häder [Sun, 4 Jun 2023 09:47:07 +0000 (11:47 +0200)]
Continued:
- the value from "misskey_offset" went into "limit" parameter, so let's rename
it to "misskey_limit"
- also misskey may return same set of results, so this need to be counted and
then the while loop aborted
- also TIME() did NOT store timestamp, we need to use time.time() again ...
Roland Häder [Sun, 4 Jun 2023 04:47:28 +0000 (06:47 +0200)]
Continued:
- there are no private variables or methods, all is public
- there is however a notation, that a leading _ indicates private access and
that you should search for getters/setters
- so when you see some code like `module._foo()` then notify them about this
Roland Häder [Sat, 3 Jun 2023 13:31:18 +0000 (15:31 +0200)]
Continued:
- added script to check if instance aka. "domain" is valid, not blacklisted
and not already registered
- in any of these conditions, an other status code is returned
Roland Häder [Fri, 2 Jun 2023 13:44:15 +0000 (15:44 +0200)]
Continued:
- first thanks to `activitypub-trolls.cf` we have tons of registered
"instances" which needs to be fetched and then these trolls need to be
ignored
- then commented out some debug lines
- also fixed some code
- also let */? pass as obsfucations
Roland Häder [Fri, 2 Jun 2023 09:55:26 +0000 (11:55 +0200)]
Continued:
- renamed update_nodeinfos() to update_instance_data()
- also only delete pending instance data if UPDATE statement has updated
something (at least the timestamps SHOULD cause a row update)
Roland Häder [Fri, 2 Jun 2023 09:52:14 +0000 (11:52 +0200)]
Continued:
- need to check 'peer' if type 'None'
- 48 hours is enough for checking for new instances, so fast no new instance
will be created
- updating 'last_updated' now moved to update_nodeinfos()