]> git.mxchange.org Git - fba.git/log
fba.git
17 months agoContinued:
Roland Häder [Sun, 11 Jun 2023 13:20:43 +0000 (15:20 +0200)]
Continued:
- removed another broad except block
- added missing variable initialization
- there is no 'domain', aka. instance here
- fixed bare-except, you can use 'if foo in bar' instead

17 months agoContinued:
Roland Häder [Sun, 11 Jun 2023 12:49:07 +0000 (14:49 +0200)]
Continued:
- handle typical network exceptions here, too
- don't sys.exit() here anymore, exit function properly
- don't 'break' here, there was no loop, better return proper error code
- fixed a few problems, found by pylint

17 months agoContinued:
Roland Häder [Sun, 11 Jun 2023 12:21:57 +0000 (14:21 +0200)]
Continued:
- fetch[_wellknown]_nodeinfo() should return same dict with at least
  'status_code' and 'json' on success and 'error_message' on failure

17 months agoContinued:
Roland Häder [Sun, 11 Jun 2023 11:26:17 +0000 (13:26 +0200)]
Continued:
- if dict 'error' cannot be handled (their keys) then raise an exception
- put all type(foo) into single quotes
- commented out debug line

17 months agoContinued:
Roland Häder [Sun, 11 Jun 2023 02:53:42 +0000 (04:53 +0200)]
Continued:
- also need to put a try/except block around csrf.determine() invocations

17 months agoContinued:
Roland Häder [Sun, 11 Jun 2023 02:48:43 +0000 (04:48 +0200)]
Continued:
- forgot to copy-paste error_message check for "blocked"
- also needed to take fetched['json'] for fetched rows

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 22:35:17 +0000 (00:35 +0200)]
Continued:
- some misskey (Chinese) instance returned broken UTF-8, so I had to catch
  UnicodeEncodeError, too

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 22:22:15 +0000 (00:22 +0200)]
Continued:
- both ['error_message'] and ['json']['federated_instances'] is extremely rare

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 18:24:22 +0000 (20:24 +0200)]
Continued:
- at least show the exception name

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 17:56:16 +0000 (19:56 +0200)]
Continued:
- ops, don't copy whole HTML to last_error_message
- also need to initialize local variable 'peers' first

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 17:40:37 +0000 (19:40 +0200)]
Continued:
- no parameter for 'timeout' (here: 'headers' wasn't handled over)

17 months agoWIP(?):
Roland Häder [Sat, 10 Jun 2023 17:32:52 +0000 (19:32 +0200)]
WIP(?):
- moved csrf.determine() of of central functions, it was causing to much CSRF
  checks and slowed down all

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 17:05:40 +0000 (19:05 +0200)]
Continued:
- ops wrong fix

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 16:59:09 +0000 (18:59 +0200)]
Continued:
- DEBUG level + commented out is enough here

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 16:50:26 +0000 (18:50 +0200)]
Continued:
- caught more specific exception and not broad BaseException (still some are
  left)
- renamed variable blocks -> blocking to avoid confusion with imported
  fba.blocks

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 15:57:54 +0000 (17:57 +0200)]
Continued:
- also here need to handle error_message
- fetched['json'] does now contain the rows

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 15:51:55 +0000 (17:51 +0200)]
Continued:
- whyever, but a list is returned, not a dict

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 15:49:35 +0000 (17:49 +0200)]
Continued:
- also catch this exception here, avoiding a lot of more requests to unreachable websites

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 14:38:05 +0000 (16:38 +0200)]
Continued:
- data['json'] now contains the reply

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 14:32:39 +0000 (16:32 +0200)]
Continued:
- still a requests.model.Response might be handled over

17 months agoWIP:
Roland Häder [Sat, 10 Jun 2023 14:29:43 +0000 (16:29 +0200)]
WIP:
- introduced network.fetch_api_url() which takes a whole URL (not domain/path
  separated) and returns it's reply in a json_reply dictionary as both other
  API functions are handling it

17 months agoWIP:
Roland Häder [Sat, 10 Jun 2023 14:14:54 +0000 (16:14 +0200)]
WIP:
- more refacturing away from "generic except blocks"

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 13:27:05 +0000 (15:27 +0200)]
Continued:
- introduced get_json_api() which gives back a dictionary,
- 'status_code' in it is always set, so better test on 'error_message'

17 months agoFixed:
Roland Häder [Sat, 10 Jun 2023 12:06:45 +0000 (14:06 +0200)]
Fixed:
- had to add a 'import reqto' only here to make Python happy
- looks like something in Python internal mapping or so is broken?

17 months agoContinued: broken/missing-import-reqto
Roland Häder [Sat, 10 Jun 2023 11:42:20 +0000 (13:42 +0200)]
Continued:
- first 'rows' was re-assigned with new values, which is typical for Python
- yet still this hasn't fixed the recursion error

17 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 00:10:02 +0000 (02:10 +0200)]
Continued:
- removed redundant initialization of 'blockdict'
- removed local variable 'csrf'

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 22:51:24 +0000 (00:51 +0200)]
Continued:
- updated execution in systemd service file

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 22:49:39 +0000 (00:49 +0200)]
Continued:
- explained why the original website isn't reflecting this code

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 22:32:38 +0000 (00:32 +0200)]
Continued:
- updated README.md

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 21:29:18 +0000 (23:29 +0200)]
Continued:
- removed remains of code around pending_errors dictionary
- rewrote it towards instances.update_last_error()
- also rewrote/removed some try/except blocks

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 18:55:58 +0000 (20:55 +0200)]
Continued:
- markdown is another requirement

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 18:42:53 +0000 (20:42 +0200)]
Continued:
- ops, double commented out
- also peertube may need debugging later on

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 17:52:11 +0000 (19:52 +0200)]
Continued:
- renamed fba/federation/ -> fba/networks/ to make room for ...
- added fba.federation.py module which has generic functions for federation
- introduced fba/locking.py module
- renamed aqcuire_lock() -> aqcuire()
- added locking.release()
- introduced fba/helpers/dict.py and tidyup.py
- renamed tidyu_*() to tidyup.*()
- this all-in-all was an attempt to fix cyclic imports, still some are left

17 months agoWIP:
Roland Häder [Fri, 9 Jun 2023 11:38:13 +0000 (13:38 +0200)]
WIP:
- avoid these generic catches all together and try to catch them individually
  where it makes sense

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 10:08:31 +0000 (12:08 +0200)]
Continued:
- not deleted here ...

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 10:07:38 +0000 (12:07 +0200)]
Continued:
- formatted for shorter lines

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 10:04:53 +0000 (12:04 +0200)]
Continued:
- ops, have forgotten to move this function to own module

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 09:57:40 +0000 (11:57 +0200)]
Continued:
- renaming 'exception' to 'exc' didn't fix the problem, I thought here about
  overlapping/overwriting definitions
- but pylint3 would have it shown anyway

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 09:24:33 +0000 (11:24 +0200)]
Continued:
- eventlet is nowhere used
- need to import jinja2 templating like this

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 08:38:54 +0000 (10:38 +0200)]
Continued:
- fixed logger message

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 08:21:38 +0000 (10:21 +0200)]
Continued:
- ignored more files (tracing, coverage)
- rewrote towards fba.csrf.determine() so it will always be determined, causes
  another request unfortunately
- or can we somehow save the cookie jar ?
- logging of errors can now be disabled for debug purposes

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 04:46:23 +0000 (06:46 +0200)]
Continued:
- fixed more "Undefined variable"

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 04:28:39 +0000 (06:28 +0200)]
Continued:
- fixed bad indenting
- renamed json -> rows to avoid redeclaration with module 'json'
- sorted import members (standard imports first)
- removed unused imports
- fixed "Redefining name 'headers' from outer scope"

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 04:12:38 +0000 (06:12 +0200)]
Continued:
- renamed originator -> command

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 03:45:04 +0000 (05:45 +0200)]
Continued:
- moved headers,api_headers to fba.network module
- fixed more references

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 03:32:23 +0000 (05:32 +0200)]
Continued:
- the repository is more up-to-date, let's fetch this instead
- had to parse the markdown code back to HTML to be able to traverse with
  existing code over it

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 02:42:57 +0000 (04:42 +0200)]
Continued:
- removed some redundant CSS
- also made both :hover color the same
- td's text alignment is now left

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 02:28:29 +0000 (04:28 +0200)]
Continued:
- catch BaseException (very generic catch, but wanted here)

17 months agoFixed some issues found by pylint:
Roland Häder [Fri, 9 Jun 2023 00:50:37 +0000 (02:50 +0200)]
Fixed some issues found by pylint:
- added pylint.rc file (all checks are enabled)
- '!= None' should be 'is not None'
- '== None' should be 'is None'
- '!= type' should be 'not isinstance(var, type)'
- fixed some 'unused variable'
- fixed 'duplicate definition'
- fixed 'invalid name'
- fixed f"foo" with no {var} in it
- renamed more variables

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 00:35:07 +0000 (02:35 +0200)]
Continued:
- moved wrapper functions around reqto.get|post() to module fba/network.py
- renamed get_response() -> fetch_response()

17 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 00:14:39 +0000 (02:14 +0200)]
Continued:
- github.com is not an instance, so let's not confuse some code somewhere

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 23:58:53 +0000 (01:58 +0200)]
Continued:
- added command 'fetch_federater' which fetches a CSV file from github
- if software is null, attempt to determine it
- renamed get_url() to fetch_url()

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 22:15:29 +0000 (00:15 +0200)]
Continued:
- "blocked" needs to be "reject" to be in align with other software

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 19:18:54 +0000 (21:18 +0200)]
Continued:
- added missing check for reason != None
- empty strings are now NoneType, too

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:52:18 +0000 (20:52 +0200)]
Continued:
- explained error code 999
- added missing header for this scoreboard

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:35:52 +0000 (20:35 +0200)]
Continued:
- moved fba.fetch_misskey_blocks() to misskey.fetch_blocks()
- avoided calling tidyup_reason() on NoneType

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:24:46 +0000 (20:24 +0200)]
Continued:
- gotosocial has no open API for instance blocks

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:23:01 +0000 (20:23 +0200)]
Continued:
- ops, oversaw this one

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:03:43 +0000 (20:03 +0200)]
Continued:
- some block reasons have a long domain list (which can be taken as
  instance names, too) with no space after a comma
- this causes the web page to be very(!) wide
- added missing "import inspect"
- improved some debug messages

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 17:39:35 +0000 (19:39 +0200)]
Continued:
- added "TOP 30" of error codes (excluding 200)

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 17:07:03 +0000 (19:07 +0200)]
Continued:
- make table border visible

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 16:50:52 +0000 (18:50 +0200)]
Continued:
- also tidyup reason after fetching from instance
- moved validating domain to upper if/elif blocks
- re-color website to more friendlier colors
- rewrote some debug lines

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 14:54:09 +0000 (16:54 +0200)]
Continued:
- "get" should be used only for getters
- removed variable "get_peers_url" which was used only once in code

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 14:43:04 +0000 (16:43 +0200)]
Continued:
- moved templates to templates/views/
- introduced base.html which should be extended by view templates
- TOP x lists now have an own view template

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 06:01:00 +0000 (08:01 +0200)]
Continued:
- moved more instances-related functions to module fba.instances
- also cut of /profile/ and /users/ from paths

17 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 05:38:33 +0000 (07:38 +0200)]
Continued:
- moved and renamed fba.add_instance() to instances.add()

17 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 17:08:09 +0000 (19:08 +0200)]
Continued:
- moved gotosocial part to fba/federation/gotosocial.py
- fixed duplicate variable names

17 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 12:19:14 +0000 (14:19 +0200)]
Continued:
- find_domains() is in module 'fba'

17 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 01:27:18 +0000 (03:27 +0200)]
Continued:
- needed to rename blocks -> blocklist to avoid mixing with fba.blocks

17 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 00:54:15 +0000 (02:54 +0200)]
Continued:
- parameter 'software' is no longer available

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 22:26:21 +0000 (00:26 +0200)]
Continued:
- commented out debug lines
- added commented-out lines, too ;-)
- tidyup software name

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 22:21:02 +0000 (00:21 +0200)]
Continued:
- don't link to invalid domain names, if invalid a HTTPException is raised

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 21:28:40 +0000 (23:28 +0200)]
Continued:
- ops, I have not removed this one

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 20:11:46 +0000 (22:11 +0200)]
Continued:
- renamed package 'network' -> 'federation'

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 20:07:09 +0000 (22:07 +0200)]
Continued:
- moved blacklist-related stuff to fba/blacklist.py
- and now, the word 'blacklist' is not racism, words are innocent!

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 19:55:39 +0000 (21:55 +0200)]
Continued:
- moved functions for table 'blocks' to own module
- renamed some of them
- fixed missing reference to fba.api_headers

17 months agoWIP:
Roland Häder [Tue, 6 Jun 2023 18:43:41 +0000 (20:43 +0200)]
WIP:
- moved network-specific function to their own fba/network/<network>.py module
- renamed blocker -> domain in network-specific functions (it is 'domain' in
  the end)

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:53:36 +0000 (12:53 +0200)]
Continued:
- maybe this floods your output a little ...
- but still I need to find missing translations

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:48:08 +0000 (12:48 +0200)]
Continued:
- don't use tidyup_domain() on header text
- also need to check /about/more - they often "hide" there the block list

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:33:47 +0000 (12:33 +0200)]
Continued:
- moved timestamp formatting to configuration file
- added navigation back to /

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:16:56 +0000 (12:16 +0200)]
Continued:
- renamed result <-> blocks
- added config entry 'hostname' which you should change to where you have
  installed it, e.g. mine is 'fba.mxchange.org' (password-protected)

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 09:44:56 +0000 (11:44 +0200)]
Continued:
- added more for <title> so search engines have more pages to index and not
  combine them into one result only
- still we need paging support here

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 09:35:11 +0000 (11:35 +0200)]
Continued:
- `inspect.currentframe().f_code.co_name` is one LARGE way for finding out
  current function's name but it does not involve any private functions
  being called publicly
- still PHP has __FUNCTION__ for this purpose ...

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 08:43:08 +0000 (10:43 +0200)]
Continued:
- sys.argv[0] is now always ./fba.py, so let's take the function's name
  instead

17 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 07:19:16 +0000 (09:19 +0200)]
Continued:
- renamed has_element() to has_key()
- fixed bad handling of has_key()
- also need to count added names and exit loop on zero

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 23:58:09 +0000 (01:58 +0200)]
Continued:
- only module 'boot' is needed here

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 23:18:13 +0000 (01:18 +0200)]
Continued:
- introduced get_url() which uses urllib.parse() to wrap invocation around
  get_response() nicely
- so all reqto.get|post() invocations are now centralized for later easy
  expansion with SOCKS proxy support

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:41:07 +0000 (23:41 +0200)]
Continued:
- add argument --single to commands.fetch_instances() which only fetches given
  instance
- int() doesn't "magically" turn NoneType to 0, PHP would do it, so let's handle
  this here by ourself

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:32:04 +0000 (23:32 +0200)]
Continued:
- exit with returned status code

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:22:29 +0000 (23:22 +0200)]
Continued:
- prog is the script's name by default which is okay
- added epilog

17 months agoFixed:
Roland Häder [Mon, 5 Jun 2023 21:16:44 +0000 (23:16 +0200)]
Fixed:
- --domain is required for command 'fetch_instances'

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:11:29 +0000 (23:11 +0200)]
Continued:
- renamed command.py -> commands.py

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 20:09:42 +0000 (22:09 +0200)]
Continued:
- introduced argparse which is a more flexible way of handling command-line
  arguments
- moved all commands to fba/command.py you can now access them through
  $ ./fba.py <command>
- please use --help to see which commands are all supported, you can also use
  it on a single command to get all supported arguments

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 09:53:16 +0000 (11:53 +0200)]
Continued:
- moved acquiring lock to a place when a domain is supposed to be
  added
- added missing fba.tidyup_reason()

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 08:30:38 +0000 (10:30 +0200)]
Continued:
- origin=NULL was shown as "None", so let's exclude this

17 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 07:31:18 +0000 (09:31 +0200)]
Continued:
- some APIs return the same result at the end all over again
- so we need to check if the domain is already part of the created list
- for that purpose has_element() is introduced

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 21:32:58 +0000 (23:32 +0200)]
Continued:
- renamed 'res' to 'response'
- catch exception, log error and raise it again

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 21:17:38 +0000 (23:17 +0200)]
Continued:
- introduced fba.get_response()

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 19:08:32 +0000 (21:08 +0200)]
Continued:
- invocation, not calling ;-)

17 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 14:06:37 +0000 (16:06 +0200)]
Continued:
- retrieving nodeinfo shouldn't take that long