]> git.mxchange.org Git - fba.git/log
fba.git
15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 17:56:16 +0000 (19:56 +0200)]
Continued:
- ops, don't copy whole HTML to last_error_message
- also need to initialize local variable 'peers' first

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 17:40:37 +0000 (19:40 +0200)]
Continued:
- no parameter for 'timeout' (here: 'headers' wasn't handled over)

15 months agoWIP(?):
Roland Häder [Sat, 10 Jun 2023 17:32:52 +0000 (19:32 +0200)]
WIP(?):
- moved csrf.determine() of of central functions, it was causing to much CSRF
  checks and slowed down all

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 17:05:40 +0000 (19:05 +0200)]
Continued:
- ops wrong fix

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 16:59:09 +0000 (18:59 +0200)]
Continued:
- DEBUG level + commented out is enough here

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 16:50:26 +0000 (18:50 +0200)]
Continued:
- caught more specific exception and not broad BaseException (still some are
  left)
- renamed variable blocks -> blocking to avoid confusion with imported
  fba.blocks

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 15:57:54 +0000 (17:57 +0200)]
Continued:
- also here need to handle error_message
- fetched['json'] does now contain the rows

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 15:51:55 +0000 (17:51 +0200)]
Continued:
- whyever, but a list is returned, not a dict

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 15:49:35 +0000 (17:49 +0200)]
Continued:
- also catch this exception here, avoiding a lot of more requests to unreachable websites

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 14:38:05 +0000 (16:38 +0200)]
Continued:
- data['json'] now contains the reply

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 14:32:39 +0000 (16:32 +0200)]
Continued:
- still a requests.model.Response might be handled over

15 months agoWIP:
Roland Häder [Sat, 10 Jun 2023 14:29:43 +0000 (16:29 +0200)]
WIP:
- introduced network.fetch_api_url() which takes a whole URL (not domain/path
  separated) and returns it's reply in a json_reply dictionary as both other
  API functions are handling it

15 months agoWIP:
Roland Häder [Sat, 10 Jun 2023 14:14:54 +0000 (16:14 +0200)]
WIP:
- more refacturing away from "generic except blocks"

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 13:27:05 +0000 (15:27 +0200)]
Continued:
- introduced get_json_api() which gives back a dictionary,
- 'status_code' in it is always set, so better test on 'error_message'

15 months agoFixed:
Roland Häder [Sat, 10 Jun 2023 12:06:45 +0000 (14:06 +0200)]
Fixed:
- had to add a 'import reqto' only here to make Python happy
- looks like something in Python internal mapping or so is broken?

15 months agoContinued: broken/missing-import-reqto
Roland Häder [Sat, 10 Jun 2023 11:42:20 +0000 (13:42 +0200)]
Continued:
- first 'rows' was re-assigned with new values, which is typical for Python
- yet still this hasn't fixed the recursion error

15 months agoContinued:
Roland Häder [Sat, 10 Jun 2023 00:10:02 +0000 (02:10 +0200)]
Continued:
- removed redundant initialization of 'blockdict'
- removed local variable 'csrf'

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 22:51:24 +0000 (00:51 +0200)]
Continued:
- updated execution in systemd service file

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 22:49:39 +0000 (00:49 +0200)]
Continued:
- explained why the original website isn't reflecting this code

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 22:32:38 +0000 (00:32 +0200)]
Continued:
- updated README.md

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 21:29:18 +0000 (23:29 +0200)]
Continued:
- removed remains of code around pending_errors dictionary
- rewrote it towards instances.update_last_error()
- also rewrote/removed some try/except blocks

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 18:55:58 +0000 (20:55 +0200)]
Continued:
- markdown is another requirement

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 18:42:53 +0000 (20:42 +0200)]
Continued:
- ops, double commented out
- also peertube may need debugging later on

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 17:52:11 +0000 (19:52 +0200)]
Continued:
- renamed fba/federation/ -> fba/networks/ to make room for ...
- added fba.federation.py module which has generic functions for federation
- introduced fba/locking.py module
- renamed aqcuire_lock() -> aqcuire()
- added locking.release()
- introduced fba/helpers/dict.py and tidyup.py
- renamed tidyu_*() to tidyup.*()
- this all-in-all was an attempt to fix cyclic imports, still some are left

15 months agoWIP:
Roland Häder [Fri, 9 Jun 2023 11:38:13 +0000 (13:38 +0200)]
WIP:
- avoid these generic catches all together and try to catch them individually
  where it makes sense

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 10:08:31 +0000 (12:08 +0200)]
Continued:
- not deleted here ...

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 10:07:38 +0000 (12:07 +0200)]
Continued:
- formatted for shorter lines

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 10:04:53 +0000 (12:04 +0200)]
Continued:
- ops, have forgotten to move this function to own module

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 09:57:40 +0000 (11:57 +0200)]
Continued:
- renaming 'exception' to 'exc' didn't fix the problem, I thought here about
  overlapping/overwriting definitions
- but pylint3 would have it shown anyway

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 09:24:33 +0000 (11:24 +0200)]
Continued:
- eventlet is nowhere used
- need to import jinja2 templating like this

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 08:38:54 +0000 (10:38 +0200)]
Continued:
- fixed logger message

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 08:21:38 +0000 (10:21 +0200)]
Continued:
- ignored more files (tracing, coverage)
- rewrote towards fba.csrf.determine() so it will always be determined, causes
  another request unfortunately
- or can we somehow save the cookie jar ?
- logging of errors can now be disabled for debug purposes

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 04:46:23 +0000 (06:46 +0200)]
Continued:
- fixed more "Undefined variable"

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 04:28:39 +0000 (06:28 +0200)]
Continued:
- fixed bad indenting
- renamed json -> rows to avoid redeclaration with module 'json'
- sorted import members (standard imports first)
- removed unused imports
- fixed "Redefining name 'headers' from outer scope"

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 04:12:38 +0000 (06:12 +0200)]
Continued:
- renamed originator -> command

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 03:45:04 +0000 (05:45 +0200)]
Continued:
- moved headers,api_headers to fba.network module
- fixed more references

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 03:32:23 +0000 (05:32 +0200)]
Continued:
- the repository is more up-to-date, let's fetch this instead
- had to parse the markdown code back to HTML to be able to traverse with
  existing code over it

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 02:42:57 +0000 (04:42 +0200)]
Continued:
- removed some redundant CSS
- also made both :hover color the same
- td's text alignment is now left

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 02:28:29 +0000 (04:28 +0200)]
Continued:
- catch BaseException (very generic catch, but wanted here)

15 months agoFixed some issues found by pylint:
Roland Häder [Fri, 9 Jun 2023 00:50:37 +0000 (02:50 +0200)]
Fixed some issues found by pylint:
- added pylint.rc file (all checks are enabled)
- '!= None' should be 'is not None'
- '== None' should be 'is None'
- '!= type' should be 'not isinstance(var, type)'
- fixed some 'unused variable'
- fixed 'duplicate definition'
- fixed 'invalid name'
- fixed f"foo" with no {var} in it
- renamed more variables

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 00:35:07 +0000 (02:35 +0200)]
Continued:
- moved wrapper functions around reqto.get|post() to module fba/network.py
- renamed get_response() -> fetch_response()

15 months agoContinued:
Roland Häder [Fri, 9 Jun 2023 00:14:39 +0000 (02:14 +0200)]
Continued:
- github.com is not an instance, so let's not confuse some code somewhere

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 23:58:53 +0000 (01:58 +0200)]
Continued:
- added command 'fetch_federater' which fetches a CSV file from github
- if software is null, attempt to determine it
- renamed get_url() to fetch_url()

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 22:15:29 +0000 (00:15 +0200)]
Continued:
- "blocked" needs to be "reject" to be in align with other software

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 19:18:54 +0000 (21:18 +0200)]
Continued:
- added missing check for reason != None
- empty strings are now NoneType, too

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:52:18 +0000 (20:52 +0200)]
Continued:
- explained error code 999
- added missing header for this scoreboard

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:35:52 +0000 (20:35 +0200)]
Continued:
- moved fba.fetch_misskey_blocks() to misskey.fetch_blocks()
- avoided calling tidyup_reason() on NoneType

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:24:46 +0000 (20:24 +0200)]
Continued:
- gotosocial has no open API for instance blocks

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:23:01 +0000 (20:23 +0200)]
Continued:
- ops, oversaw this one

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 18:03:43 +0000 (20:03 +0200)]
Continued:
- some block reasons have a long domain list (which can be taken as
  instance names, too) with no space after a comma
- this causes the web page to be very(!) wide
- added missing "import inspect"
- improved some debug messages

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 17:39:35 +0000 (19:39 +0200)]
Continued:
- added "TOP 30" of error codes (excluding 200)

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 17:07:03 +0000 (19:07 +0200)]
Continued:
- make table border visible

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 16:50:52 +0000 (18:50 +0200)]
Continued:
- also tidyup reason after fetching from instance
- moved validating domain to upper if/elif blocks
- re-color website to more friendlier colors
- rewrote some debug lines

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 14:54:09 +0000 (16:54 +0200)]
Continued:
- "get" should be used only for getters
- removed variable "get_peers_url" which was used only once in code

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 14:43:04 +0000 (16:43 +0200)]
Continued:
- moved templates to templates/views/
- introduced base.html which should be extended by view templates
- TOP x lists now have an own view template

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 06:01:00 +0000 (08:01 +0200)]
Continued:
- moved more instances-related functions to module fba.instances
- also cut of /profile/ and /users/ from paths

15 months agoContinued:
Roland Häder [Thu, 8 Jun 2023 05:38:33 +0000 (07:38 +0200)]
Continued:
- moved and renamed fba.add_instance() to instances.add()

15 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 17:08:09 +0000 (19:08 +0200)]
Continued:
- moved gotosocial part to fba/federation/gotosocial.py
- fixed duplicate variable names

15 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 12:19:14 +0000 (14:19 +0200)]
Continued:
- find_domains() is in module 'fba'

15 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 01:27:18 +0000 (03:27 +0200)]
Continued:
- needed to rename blocks -> blocklist to avoid mixing with fba.blocks

15 months agoContinued:
Roland Häder [Wed, 7 Jun 2023 00:54:15 +0000 (02:54 +0200)]
Continued:
- parameter 'software' is no longer available

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 22:26:21 +0000 (00:26 +0200)]
Continued:
- commented out debug lines
- added commented-out lines, too ;-)
- tidyup software name

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 22:21:02 +0000 (00:21 +0200)]
Continued:
- don't link to invalid domain names, if invalid a HTTPException is raised

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 21:28:40 +0000 (23:28 +0200)]
Continued:
- ops, I have not removed this one

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 20:11:46 +0000 (22:11 +0200)]
Continued:
- renamed package 'network' -> 'federation'

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 20:07:09 +0000 (22:07 +0200)]
Continued:
- moved blacklist-related stuff to fba/blacklist.py
- and now, the word 'blacklist' is not racism, words are innocent!

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 19:55:39 +0000 (21:55 +0200)]
Continued:
- moved functions for table 'blocks' to own module
- renamed some of them
- fixed missing reference to fba.api_headers

15 months agoWIP:
Roland Häder [Tue, 6 Jun 2023 18:43:41 +0000 (20:43 +0200)]
WIP:
- moved network-specific function to their own fba/network/<network>.py module
- renamed blocker -> domain in network-specific functions (it is 'domain' in
  the end)

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:53:36 +0000 (12:53 +0200)]
Continued:
- maybe this floods your output a little ...
- but still I need to find missing translations

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:48:08 +0000 (12:48 +0200)]
Continued:
- don't use tidyup_domain() on header text
- also need to check /about/more - they often "hide" there the block list

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:33:47 +0000 (12:33 +0200)]
Continued:
- moved timestamp formatting to configuration file
- added navigation back to /

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 10:16:56 +0000 (12:16 +0200)]
Continued:
- renamed result <-> blocks
- added config entry 'hostname' which you should change to where you have
  installed it, e.g. mine is 'fba.mxchange.org' (password-protected)

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 09:44:56 +0000 (11:44 +0200)]
Continued:
- added more for <title> so search engines have more pages to index and not
  combine them into one result only
- still we need paging support here

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 09:35:11 +0000 (11:35 +0200)]
Continued:
- `inspect.currentframe().f_code.co_name` is one LARGE way for finding out
  current function's name but it does not involve any private functions
  being called publicly
- still PHP has __FUNCTION__ for this purpose ...

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 08:43:08 +0000 (10:43 +0200)]
Continued:
- sys.argv[0] is now always ./fba.py, so let's take the function's name
  instead

15 months agoContinued:
Roland Häder [Tue, 6 Jun 2023 07:19:16 +0000 (09:19 +0200)]
Continued:
- renamed has_element() to has_key()
- fixed bad handling of has_key()
- also need to count added names and exit loop on zero

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 23:58:09 +0000 (01:58 +0200)]
Continued:
- only module 'boot' is needed here

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 23:18:13 +0000 (01:18 +0200)]
Continued:
- introduced get_url() which uses urllib.parse() to wrap invocation around
  get_response() nicely
- so all reqto.get|post() invocations are now centralized for later easy
  expansion with SOCKS proxy support

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:41:07 +0000 (23:41 +0200)]
Continued:
- add argument --single to commands.fetch_instances() which only fetches given
  instance
- int() doesn't "magically" turn NoneType to 0, PHP would do it, so let's handle
  this here by ourself

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:32:04 +0000 (23:32 +0200)]
Continued:
- exit with returned status code

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:22:29 +0000 (23:22 +0200)]
Continued:
- prog is the script's name by default which is okay
- added epilog

15 months agoFixed:
Roland Häder [Mon, 5 Jun 2023 21:16:44 +0000 (23:16 +0200)]
Fixed:
- --domain is required for command 'fetch_instances'

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 21:11:29 +0000 (23:11 +0200)]
Continued:
- renamed command.py -> commands.py

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 20:09:42 +0000 (22:09 +0200)]
Continued:
- introduced argparse which is a more flexible way of handling command-line
  arguments
- moved all commands to fba/command.py you can now access them through
  $ ./fba.py <command>
- please use --help to see which commands are all supported, you can also use
  it on a single command to get all supported arguments

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 09:53:16 +0000 (11:53 +0200)]
Continued:
- moved acquiring lock to a place when a domain is supposed to be
  added
- added missing fba.tidyup_reason()

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 08:30:38 +0000 (10:30 +0200)]
Continued:
- origin=NULL was shown as "None", so let's exclude this

15 months agoContinued:
Roland Häder [Mon, 5 Jun 2023 07:31:18 +0000 (09:31 +0200)]
Continued:
- some APIs return the same result at the end all over again
- so we need to check if the domain is already part of the created list
- for that purpose has_element() is introduced

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 21:32:58 +0000 (23:32 +0200)]
Continued:
- renamed 'res' to 'response'
- catch exception, log error and raise it again

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 21:17:38 +0000 (23:17 +0200)]
Continued:
- introduced fba.get_response()

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 19:08:32 +0000 (21:08 +0200)]
Continued:
- invocation, not calling ;-)

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 14:06:37 +0000 (16:06 +0200)]
Continued:
- retrieving nodeinfo shouldn't take that long

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 13:37:05 +0000 (15:37 +0200)]
Continued:
- package/module of JSONDecodeError is json.decoder.JSONDecodeError

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 12:54:47 +0000 (14:54 +0200)]
Continued:
- introduced json_from_response() which handles decoding errors, e.g. when
  a server has returned a HTML instead of a JSON which is caused by improper
  error handling

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 11:32:55 +0000 (13:32 +0200)]
Continued:
- trim off last dot in host/domain name
- only release lock when acquired, e.g. check_instance.py doesn't need a lock

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 11:06:23 +0000 (13:06 +0200)]
Continued:
- renamed functions in 'cache' module as they are already in proper module
- tried to prevent error if an empty (0 Byte) result is returned

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 09:47:07 +0000 (11:47 +0200)]
Continued:
- the value from "misskey_offset" went into "limit" parameter, so let's rename
  it to "misskey_limit"
- also misskey may return same set of results, so this need to be counted and
  then the while loop aborted
- also TIME() did NOT store timestamp, we need to use time.time() again ...

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 09:07:05 +0000 (11:07 +0200)]
Continued:
- it is without slashes

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 04:47:28 +0000 (06:47 +0200)]
Continued:
- there are no private variables or methods, all is public
- there is however a notation, that a leading _ indicates private access and
  that you should search for getters/setters
- so when you see some code like `module._foo()` then notify them about this

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 04:42:46 +0000 (06:42 +0200)]
Continued:
- let the config value be accessed over config.get()

15 months agoContinued:
Roland Häder [Sun, 4 Jun 2023 04:32:58 +0000 (06:32 +0200)]
Continued:
- move config-related stuff to new module 'config'