]> git.mxchange.org Git - fba.git/log
fba.git
17 months agoWIP:
Roland Häder [Fri, 9 Jun 2023 11:38:13 +0000 (13:38 +0200)]
WIP:
- avoid these generic catches all together and try to catch them individually
  where it makes sense

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 10:08:31 +0000 (12:08 +0200)]
Continued:
- not deleted here ...

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 10:07:38 +0000 (12:07 +0200)]
Continued:
- formatted for shorter lines

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 10:04:53 +0000 (12:04 +0200)]
Continued:
- ops, have forgotten to move this function to own module

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 09:57:40 +0000 (11:57 +0200)]
Continued:
- renaming 'exception' to 'exc' didn't fix the problem, I thought here about
  overlapping/overwriting definitions
- but pylint3 would have it shown anyway

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 09:24:33 +0000 (11:24 +0200)]
Continued:
- eventlet is nowhere used
- need to import jinja2 templating like this

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 08:38:54 +0000 (10:38 +0200)]
Continued:
- fixed logger message

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 08:21:38 +0000 (10:21 +0200)]
Continued:
- ignored more files (tracing, coverage)
- rewrote towards fba.csrf.determine() so it will always be determined, causes
  another request unfortunately
- or can we somehow save the cookie jar ?
- logging of errors can now be disabled for debug purposes

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 04:46:23 +0000 (06:46 +0200)]
Continued:
- fixed more "Undefined variable"

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 04:28:39 +0000 (06:28 +0200)]
Continued:
- fixed bad indenting
- renamed json -> rows to avoid redeclaration with module 'json'
- sorted import members (standard imports first)
- removed unused imports
- fixed "Redefining name 'headers' from outer scope"

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 04:12:38 +0000 (06:12 +0200)]
Continued:
- renamed originator -> command

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 03:45:04 +0000 (05:45 +0200)]
Continued:
- moved headers,api_headers to fba.network module
- fixed more references

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 03:32:23 +0000 (05:32 +0200)]
Continued:
- the repository is more up-to-date, let's fetch this instead
- had to parse the markdown code back to HTML to be able to traverse with
  existing code over it

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 02:42:57 +0000 (04:42 +0200)]
Continued:
- removed some redundant CSS
- also made both :hover color the same
- td's text alignment is now left

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 02:28:29 +0000 (04:28 +0200)]
Continued:
- catch BaseException (very generic catch, but wanted here)

17 months agoFixed some issues found by pylint:
Roland Häder [Fri, 9 Jun 2023 00:50:37 +0000 (02:50 +0200)]
Fixed some issues found by pylint:
- added pylint.rc file (all checks are enabled)
- '!= None' should be 'is not None'
- '== None' should be 'is None'
- '!= type' should be 'not isinstance(var, type)'
- fixed some 'unused variable'
- fixed 'duplicate definition'
- fixed 'invalid name'
- fixed f"foo" with no {var} in it
- renamed more variables

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 00:35:07 +0000 (02:35 +0200)]
Continued:
- moved wrapper functions around reqto.get|post() to module fba/network.py
- renamed get_response() -> fetch_response()

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 00:14:39 +0000 (02:14 +0200)]
Continued:
- github.com is not an instance, so let's not confuse some code somewhere

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 23:58:53 +0000 (01:58 +0200)]
Continued:
- added command 'fetch_federater' which fetches a CSV file from github
- if software is null, attempt to determine it
- renamed get_url() to fetch_url()

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 22:15:29 +0000 (00:15 +0200)]
Continued:
- "blocked" needs to be "reject" to be in align with other software

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 19:18:54 +0000 (21:18 +0200)]
Continued:
- added missing check for reason != None
- empty strings are now NoneType, too

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:52:18 +0000 (20:52 +0200)]
Continued:
- explained error code 999
- added missing header for this scoreboard

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:35:52 +0000 (20:35 +0200)]
Continued:
- moved fba.fetch_misskey_blocks() to misskey.fetch_blocks()
- avoided calling tidyup_reason() on NoneType

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:24:46 +0000 (20:24 +0200)]
Continued:
- gotosocial has no open API for instance blocks

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:23:01 +0000 (20:23 +0200)]
Continued:
- ops, oversaw this one

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:03:43 +0000 (20:03 +0200)]
Continued:
- some block reasons have a long domain list (which can be taken as
  instance names, too) with no space after a comma
- this causes the web page to be very(!) wide
- added missing "import inspect"
- improved some debug messages

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 17:39:35 +0000 (19:39 +0200)]
Continued:
- added "TOP 30" of error codes (excluding 200)

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 17:07:03 +0000 (19:07 +0200)]
Continued:
- make table border visible

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 16:50:52 +0000 (18:50 +0200)]
Continued:
- also tidyup reason after fetching from instance
- moved validating domain to upper if/elif blocks
- re-color website to more friendlier colors
- rewrote some debug lines

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 14:54:09 +0000 (16:54 +0200)]
Continued:
- "get" should be used only for getters
- removed variable "get_peers_url" which was used only once in code

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 14:43:04 +0000 (16:43 +0200)]
Continued:
- moved templates to templates/views/
- introduced base.html which should be extended by view templates
- TOP x lists now have an own view template

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 06:01:00 +0000 (08:01 +0200)]
Continued:
- moved more instances-related functions to module fba.instances
- also cut of /profile/ and /users/ from paths

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 05:38:33 +0000 (07:38 +0200)]
Continued:
- moved and renamed fba.add_instance() to instances.add()

17 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 17:08:09 +0000 (19:08 +0200)]
Continued:
- moved gotosocial part to fba/federation/gotosocial.py
- fixed duplicate variable names

17 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 12:19:14 +0000 (14:19 +0200)]
Continued:
- find_domains() is in module 'fba'

17 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 01:27:18 +0000 (03:27 +0200)]
Continued:
- needed to rename blocks -> blocklist to avoid mixing with fba.blocks

17 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 00:54:15 +0000 (02:54 +0200)]
Continued:
- parameter 'software' is no longer available

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 22:26:21 +0000 (00:26 +0200)]
Continued:
- commented out debug lines
- added commented-out lines, too ;-)
- tidyup software name

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 22:21:02 +0000 (00:21 +0200)]
Continued:
- don't link to invalid domain names, if invalid a HTTPException is raised

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 21:28:40 +0000 (23:28 +0200)]
Continued:
- ops, I have not removed this one

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 20:11:46 +0000 (22:11 +0200)]
Continued:
- renamed package 'network' -> 'federation'

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 20:07:09 +0000 (22:07 +0200)]
Continued:
- moved blacklist-related stuff to fba/blacklist.py
- and now, the word 'blacklist' is not racism, words are innocent!

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 19:55:39 +0000 (21:55 +0200)]
Continued:
- moved functions for table 'blocks' to own module
- renamed some of them
- fixed missing reference to fba.api_headers

17 months agoWIP:
Roland Häder [Tue, 6 Jun 2023 18:43:41 +0000 (20:43 +0200)]
WIP:
- moved network-specific function to their own fba/network/<network>.py module
- renamed blocker -> domain in network-specific functions (it is 'domain' in
  the end)

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:53:36 +0000 (12:53 +0200)]
Continued:
- maybe this floods your output a little ...
- but still I need to find missing translations

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:48:08 +0000 (12:48 +0200)]
Continued:
- don't use tidyup_domain() on header text
- also need to check /about/more - they often "hide" there the block list

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:33:47 +0000 (12:33 +0200)]
Continued:
- moved timestamp formatting to configuration file
- added navigation back to /

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:16:56 +0000 (12:16 +0200)]
Continued:
- renamed result <-> blocks
- added config entry 'hostname' which you should change to where you have
  installed it, e.g. mine is 'fba.mxchange.org' (password-protected)

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 09:44:56 +0000 (11:44 +0200)]
Continued:
- added more for <title> so search engines have more pages to index and not
  combine them into one result only
- still we need paging support here

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 09:35:11 +0000 (11:35 +0200)]
Continued:
- `inspect.currentframe().f_code.co_name` is one LARGE way for finding out
  current function's name but it does not involve any private functions
  being called publicly
- still PHP has __FUNCTION__ for this purpose ...

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 08:43:08 +0000 (10:43 +0200)]
Continued:
- sys.argv[0] is now always ./fba.py, so let's take the function's name
  instead

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 07:19:16 +0000 (09:19 +0200)]
Continued:
- renamed has_element() to has_key()
- fixed bad handling of has_key()
- also need to count added names and exit loop on zero

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 23:58:09 +0000 (01:58 +0200)]
Continued:
- only module 'boot' is needed here

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 23:18:13 +0000 (01:18 +0200)]
Continued:
- introduced get_url() which uses urllib.parse() to wrap invocation around
  get_response() nicely
- so all reqto.get|post() invocations are now centralized for later easy
  expansion with SOCKS proxy support

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:41:07 +0000 (23:41 +0200)]
Continued:
- add argument --single to commands.fetch_instances() which only fetches given
  instance
- int() doesn't "magically" turn NoneType to 0, PHP would do it, so let's handle
  this here by ourself

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:32:04 +0000 (23:32 +0200)]
Continued:
- exit with returned status code

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:22:29 +0000 (23:22 +0200)]
Continued:
- prog is the script's name by default which is okay
- added epilog

17 months agoFixed:
Roland Häder [Mon, 5 Jun 2023 21:16:44 +0000 (23:16 +0200)]
Fixed:
- --domain is required for command 'fetch_instances'

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:11:29 +0000 (23:11 +0200)]
Continued:
- renamed command.py -> commands.py

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 20:09:42 +0000 (22:09 +0200)]
Continued:
- introduced argparse which is a more flexible way of handling command-line
  arguments
- moved all commands to fba/command.py you can now access them through
  $ ./fba.py <command>
- please use --help to see which commands are all supported, you can also use
  it on a single command to get all supported arguments

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 09:53:16 +0000 (11:53 +0200)]
Continued:
- moved acquiring lock to a place when a domain is supposed to be
  added
- added missing fba.tidyup_reason()

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 08:30:38 +0000 (10:30 +0200)]
Continued:
- origin=NULL was shown as "None", so let's exclude this

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 07:31:18 +0000 (09:31 +0200)]
Continued:
- some APIs return the same result at the end all over again
- so we need to check if the domain is already part of the created list
- for that purpose has_element() is introduced

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 21:32:58 +0000 (23:32 +0200)]
Continued:
- renamed 'res' to 'response'
- catch exception, log error and raise it again

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 21:17:38 +0000 (23:17 +0200)]
Continued:
- introduced fba.get_response()

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 19:08:32 +0000 (21:08 +0200)]
Continued:
- invocation, not calling ;-)

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 14:06:37 +0000 (16:06 +0200)]
Continued:
- retrieving nodeinfo shouldn't take that long

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 13:37:05 +0000 (15:37 +0200)]
Continued:
- package/module of JSONDecodeError is json.decoder.JSONDecodeError

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 12:54:47 +0000 (14:54 +0200)]
Continued:
- introduced json_from_response() which handles decoding errors, e.g. when
  a server has returned a HTML instead of a JSON which is caused by improper
  error handling

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 11:32:55 +0000 (13:32 +0200)]
Continued:
- trim off last dot in host/domain name
- only release lock when acquired, e.g. check_instance.py doesn't need a lock

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 11:06:23 +0000 (13:06 +0200)]
Continued:
- renamed functions in 'cache' module as they are already in proper module
- tried to prevent error if an empty (0 Byte) result is returned

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 09:47:07 +0000 (11:47 +0200)]
Continued:
- the value from "misskey_offset" went into "limit" parameter, so let's rename
  it to "misskey_limit"
- also misskey may return same set of results, so this need to be counted and
  then the while loop aborted
- also TIME() did NOT store timestamp, we need to use time.time() again ...

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 09:07:05 +0000 (11:07 +0200)]
Continued:
- it is without slashes

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 04:47:28 +0000 (06:47 +0200)]
Continued:
- there are no private variables or methods, all is public
- there is however a notation, that a leading _ indicates private access and
  that you should search for getters/setters
- so when you see some code like `module._foo()` then notify them about this

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 04:42:46 +0000 (06:42 +0200)]
Continued:
- let the config value be accessed over config.get()

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 04:32:58 +0000 (06:32 +0200)]
Continued:
- move config-related stuff to new module 'config'

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 03:07:20 +0000 (05:07 +0200)]
Continued:
- committing every single INSERT INTO will cause a lot of I/O load, let's
  combine them all into a single transaction

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 00:56:56 +0000 (02:56 +0200)]
Continued:
- ops, forgot to add it here

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 23:08:13 +0000 (01:08 +0200)]
Continued:
- ops, also they need 'fba.' now

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 23:03:30 +0000 (01:03 +0200)]
Continued:
- exposed originator (Python script) Top "10"

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 22:35:37 +0000 (00:35 +0200)]
Continued:
- moved instance-related functions and module-wide variables to 'instances.py'

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 19:54:12 +0000 (21:54 +0200)]
Continued:
- renamed tidyup() to tidyup_domains()
- so tidyup_reason() can come (WIP)

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 17:20:12 +0000 (19:20 +0200)]
Continued:
- added script to fetch blocked instances by chaos.social
- introduced is_instance_blocked()

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 16:31:21 +0000 (18:31 +0200)]
Continued:
- sorted import members
- added missing 'import sys'

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 14:53:14 +0000 (16:53 +0200)]
Continued:
- invoking update_instance_data() but having no pending instance data doesn't
  make any sense

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 13:31:18 +0000 (15:31 +0200)]
Continued:
- added script to check if instance aka. "domain" is valid, not blacklisted
  and not already registered
- in any of these conditions, an other status code is returned

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 12:37:51 +0000 (14:37 +0200)]
Continued:
- fixed cache module

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 07:02:04 +0000 (09:02 +0200)]
Continued:
- 3rd parameter is not 'origin', it is 'script'

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 06:40:56 +0000 (08:40 +0200)]
Continued:
- check dictionary keys before using them, prevents KeyError being raised

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 06:37:09 +0000 (08:37 +0200)]
Continued:
- renamed lock_process() to acquire_lock()

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 06:15:19 +0000 (08:15 +0200)]
Continued:
- moved 'fba' to own folder
- splitted up file a bit: boot, cache

17 months agoContinued:
Roland Häder [Sat, 3 Jun 2023 05:50:29 +0000 (07:50 +0200)]
Continued:
- inter-process locking added, prevents nasty stuff

17 months agoContinued:
Roland Häder [Fri, 2 Jun 2023 22:36:43 +0000 (00:36 +0200)]
Continued:
- log every single problem

17 months agoContinued:
Roland Häder [Fri, 2 Jun 2023 21:02:38 +0000 (23:02 +0200)]
Continued:
- Firfox version updated
- also check if domain is already registered to avoid lots of requests

17 months agoContinued:
Roland Häder [Fri, 2 Jun 2023 21:01:20 +0000 (23:01 +0200)]
Continued:
- added Python script to fetch domains from bka.la (thank you to Kromonos)

17 months agoContinued:
Roland Häder [Fri, 2 Jun 2023 18:22:41 +0000 (20:22 +0200)]
Continued:
- added wrapper script to fetch RSS feeds from FBA installations (e.g.
  https://fba.ryona.agency/rss )

17 months agoContinued:
Roland Häder [Fri, 2 Jun 2023 17:57:12 +0000 (19:57 +0200)]
Continued:
- moved fetch_instances() to fba

17 months agoContinued:
Roland Häder [Fri, 2 Jun 2023 17:51:58 +0000 (19:51 +0200)]
Continued:
- also rewrote this

17 months agoContinued:
Roland Häder [Fri, 2 Jun 2023 14:31:44 +0000 (16:31 +0200)]
Continued:
- more debug lines added
- also fixed maybe bad function calls
- debug messages improved

17 months agoContinued:
Roland Häder [Fri, 2 Jun 2023 14:04:42 +0000 (16:04 +0200)]
Continued:
- better warn about not deobsfucated domain names