147 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
494d811846 update: If cateogories.yml only has icon:, then add name:
E.g. if _categories.yml_ is like:

```yaml
Time:
  icon: time.png
```
2025-06-16 17:13:48 +00:00
Hans-Christoph Steiner
721de49104 make_website: only parse repo_pubkey if its needed
This makes writing tests a lot easier, since the test cases no longer need
to provide a value for `repo_pubkey:`.
2025-06-08 20:12:04 +00:00
Jochen Sprickerhof
b83c1aace3
Make v2 localized config optional
Fixes:

Traceback (most recent call last):
  File "/home/fdroid/fdroidserver/fdroid", line 22, in <module>
    fdroidserver.__main__.main()
  File "/home/fdroid/fdroidserver/fdroidserver/__main__.py", line 222, in main
    raise e
  File "/home/fdroid/fdroidserver/fdroidserver/__main__.py", line 203, in main
    mod.main()
  File "/home/fdroid/fdroidserver/fdroidserver/update.py", line 2774, in main
    fdroidserver.index.make(archived_apps, archapks, repodirs[1], True)
  File "/home/fdroid/fdroidserver/fdroidserver/index.py", line 132, in make
    make_v2(
  File "/home/fdroid/fdroidserver/fdroidserver/index.py", line 728, in make_v2
    output["repo"] = v2_repo(repodict, repodir, archive)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fdroid/fdroidserver/fdroidserver/index.py", line 686, in v2_repo
    repo["icon"] = config["archive" if archive else "repo"]["icon"]
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'icon'
2025-04-28 08:38:54 +02:00
Hans-Christoph Steiner
38378ddfb7 rename v1_sort_packages to sort_package_versions 2025-03-20 15:08:32 +01:00
Hans-Christoph Steiner
8565a97465 rename functions and variables to reflect stats/ going away 2025-03-20 15:08:32 +01:00
Hans-Christoph Steiner
20569217d9 index: fail if user sets mirrors:isPrimary wrong
Really, this is not meant to be set by the user in the config.  But if they
add something harmless that'll be ignored anyway, it seems that throwing an
error is too much.  So only throw the error if it is set wrongly.
2025-03-20 11:13:09 +00:00
Hans-Christoph Steiner
031ae1103e function-local imports to limit deps for publish/signindex/gpgsign
This eliminates the need to have these installed on the signing server:

* python3-qrcode
* python3-requests

The signing server currently uses a git clone to run _fdroidserver_ and
`apt-get install` for dependencies.  This leaves "qrcode" in
"install_requires" since moving it to "extras_require" would break
`fdroid update` and `fdroid nightly` for anything that does
`pip install fdroidserver`:

https://gitlab.com/eighthave/fdroidserver/-/jobs/9386520037
2025-03-12 13:37:13 +01:00
Hans-Christoph Steiner
2f47938dbf standardize config on ruamel.yaml with a YAML 1.2 config
This is a key piece of the ongoing `PUBLISH` _config.yml_ migration. There was uneven implementation of which YAML parser to use, and that could lead to bugs where one parser might read a value one way, and a different parser will read the value a different way. I wanted to be sure that YAML 1.2 would always work.

This makes all code that handles config files use the same `ruamel.yaml` parsers.  This only touches other usages of YAML parsers when there is overlap.  This does not port all of _fdroidserver_ to `ruamel.yaml` and YAML 1.2.  The metadata files should already be YAML 1.2 anyway.

# Conflicts:
#	fdroidserver/lint.py
2025-03-10 08:48:58 +01:00
Hans-Christoph Steiner
bc118484e4 index.xml: XML escaping from Python < 3.13 which stopped converting "
index.xml is for old clients that are stuck in the past forever.  So the
format should not change at all.  Python 3.13 changed minidom so it no
longer converts " to an XML entity.
154477be72
2024-12-06 16:15:48 +01:00
Hans-Christoph Steiner
ad9b0c3294 update: only gen index.png when making index.html
* https://f-droid.org/repo does not use the index.png at all
* Some repos want to set their own index.png rather than the QR code
2024-12-04 13:13:30 +01:00
Hans-Christoph Steiner
59fcfa5dec index: download_repo_index_v2() uses mirrors
test_download_repo_index_v2_url_parsing is no longer needed, since all the
things it tested are now handled in test_download_repo_index_v2
2024-10-31 21:55:18 +01:00
Jochen Sprickerhof
81c0c9d4aa Drop stats/known_apks.txt
Use repo/index-v2.json instead.
Also fix deprecated datetime.utcnow().
2024-10-31 09:20:26 +00:00
Leo Heitmann Ruiz
0cee694c43 Remove all mentions of Flattr 2024-09-23 07:59:08 +00:00
Hans-Christoph Steiner
fbb6772e37 always use GitLab Pages, not Artifacts, with index-only mode
If the repo is too large, then it'll hit the GitLab Pages limit.  That is
basically impossible in index-only mode, so it should always use Pages.
2024-07-03 09:18:22 +00:00
Michael Pöhn
005a33732c apply review suggestions 2024-04-24 11:56:07 +00:00
Michael Pöhn
bd6afa4365
🐛 index fix: skip altstore when no IPAs present 2024-04-24 12:45:29 +02:00
Hans-Christoph Steiner
9716b5e1ab index: manual black format 2024-04-24 10:29:50 +02:00
Michael Pöhn
d00a87ed6c
🏏 alt-store index: incorporate review feedback 2024-04-23 17:28:30 +02:00
Michael Pöhn
45efb88f85
🕴️ add test for make_altstore 2024-04-23 16:05:58 +02:00
Michael Pöhn
f742799a9d
🏟️ add test for _parse_from_pbxproj
Also fix lint issues
2024-04-23 16:05:34 +02:00
Michael Pöhn
a21ed39117
🛻 move alstore index function
Move function for generating altstore index from update.py to index.py
2024-04-23 16:05:10 +02:00
Torsten Grote
faf1090214 Use fdroid.link for repo index page 2024-04-03 13:54:48 +00:00
Jochen Sprickerhof
edb2d76e25
Add unversioned whatsNew to index-v2 (Closes: #1202)
copy_triple_t_store_metadata() can read the whatsNew from an unversioned
release-notes.txt. For v2 copy this to the latest build.
2024-03-13 06:40:33 +01:00
proletarius101
947217549a feat: add servergitmirrors as a dict support 2024-02-14 16:50:39 +00:00
Hans-Christoph Steiner
a1d9d9d885 switch to loading mirrors.yml with ruamel to get YAML 1.2 support 2024-01-09 09:32:37 +00:00
Hans-Christoph Steiner
4536b130df index: add datatype checking to mirrors: and config/mirrors.yml 2023-12-07 17:25:15 +01:00
Hans-Christoph Steiner
6d541e3ef6
establish config/mirrors.yml as a way to set up mirrors
The mirrors: entry in config.yml is great for quick access and shorter
mirror lists.  Now that we are adding a lot more metadata to the mirrors,
including potentially the full text of the privacy policy, having this
in its own file should make these cases easier to manage.
2023-12-07 14:38:26 +01:00
Jochen Sprickerhof
521ff4fe17 Add default locale for generated categories
The client expects at least a name to be set.

Regression of 2c566cf6.
2023-09-21 14:14:06 +00:00
Hans-Christoph Steiner
48559ecec5 category with no apps should be ignored, even if defined in config
https://gitlab.com/fdroid/fdroidclient/-/issues/2619#note_1421280589

The test needed to change because the test index files contained category
definitions that were not ever used in the "copy tests/repo, generate java/gpg
keys, update, and gpgsign" test in tests/run-tests.
2023-09-08 10:45:53 +02:00
Hans-Christoph Steiner
2c566cf68f update: add all categories in metadata files to repo definitions
!1366 makes it so categories are now defined by the repo. Categories can be
defined in the config so that lint has a list of categories to enforce. This
also provides a place for localization and icons for the categories. The old
way of defining categories was just listing them in app metadata files. This
restores that way of functioning when using index-v2.

closes #1137
2023-09-08 10:45:51 +02:00
Hans-Christoph Steiner
4e28fad55a use constants for names used in the config dict
Hopefully this helps with the Anti-Features case confusion:
* antifeatures
* antiFeatures
* AntiFeatures
2023-07-25 14:31:09 +00:00
Hans-Christoph Steiner
190a95ab17 use consistent name for releaseChannels config file 2023-07-25 14:31:09 +00:00
Jochen Sprickerhof
2ee3cccb60 Don't modify app dict when generating v2 antifeatures
Regression of: 061ca38a
Closes: #1103
2023-06-01 17:53:34 +00:00
Hans-Christoph Steiner
061ca38afd define "string map" type for new Anti-Features explanations
closes #683
2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
024d309262 index: rename app var to app_dict, its not an App instance
Throughout the code, variables named "app" are instances of the App class.
In this case, this is related, but it is a dict not an App instance, since
it is being prepared for including in the index-v1.json.
2023-05-11 10:37:45 +02:00
Hans-Christoph Steiner
c2bc52dd85 use constant for default locale 2023-05-11 10:37:45 +02:00
Hans-Christoph Steiner
49362b5fd1 move load_locale() and file_entry() to be accessible by all modules
* load_locale -> common.load_localized_config() since common handles config
* file_entry -> metadata.file_entry() since metadata handles data format
2023-05-10 09:21:14 +02:00
Hans-Christoph Steiner
8ccc89ad4e index: fix requestsdict check order of operations
If requestsdict is None, the old logic would still check
requestsdict["uninstall"]) and crash there.
2023-05-10 09:21:14 +02:00
Hans-Christoph Steiner
2b81a66b79 App.comments is no more since !772 2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
7c692a4532 index-v2 'mirrors' fully settable from config
This lets mirrors: in config.yml be the same list-of-dicts format as it is
in index-v2.  This also includes a data format conversion to maintain the
right format for the old, unchanging index v0 and v1 formats.

#928
#1107
2023-04-27 07:53:02 +00:00
Jochen Sprickerhof
889b8cb372 Don't copy per version anti features to all versions
make_v0() modified the apps data structure to copy an anti feature for a
specific version to all versions resulting in index-v1 and -v2 to
contain wrong anti feature annotations. This patch fixes this and adds a
test that the data structure is no longer modified.

The bug shadowed bugs in the AF implementation of -v1 and -v2 resulting
in not coping the version specific data. This is corrected as well.
This is also tested now.

For -v2 the AF dict is now sorted to make the result reproducible.

Finally The NoSourceSince AF was added as a per version and overall AF
in -v1 and is now only applied as an overall AF and the test is updated
accordingly.
2023-04-04 14:20:41 +00:00
Hans-Christoph Steiner
f3e49f4bcb
download_repo_index_v2() for verified downloading of index-v2 2023-03-09 12:35:29 +01:00
Hans-Christoph Steiner
723873456d
versioned download_repo_index_v1() to be future proof 2023-03-09 12:34:52 +01:00
Jochen Sprickerhof
0f151a3779 v2: move video to translated text (Closes: #1095)
It contains a link to the actual video and can be translated.
2023-02-16 21:42:09 +01:00
linsui
5858d8fcb4 remove liberapayID 2023-01-16 17:02:56 +00:00
FC Stegerman
fc8820ff8e
index.make_v0: ensure name is a str 2023-01-10 19:40:52 +01:00
Hans-Christoph Steiner
947d94e0a9
deploy: support GitLab Job Artifacts as a mirror 2022-11-16 20:16:32 +01:00
Hans-Christoph Steiner
f24613b701
index: fix git-mirror size check for GitLab Pages
The test case had the wrong folder setup, this was confirmed on a production
repo setup.
2022-11-16 20:16:25 +01:00
Jochen Sprickerhof
1bb963d768
jarsigner: allow weak signatures
openjdk-11 11.0.17 in Debian unstable fails to verify weak signatures:

jarsigner -verbose -strict -verify tests/signindex/guardianproject.jar

         131 Fri Dec 02 20:10:00 CET 2016 META-INF/MANIFEST.MF
         252 Fri Dec 02 20:10:04 CET 2016 META-INF/1.SF
        2299 Fri Dec 02 20:10:04 CET 2016 META-INF/1.RSA
           0 Fri Dec 02 20:09:58 CET 2016 META-INF/
 m  ?  48743 Fri Dec 02 20:09:58 CET 2016 index.xml

  s = signature was verified
  m = entry is listed in manifest
  k = at least one certificate was found in keystore
  ? = unsigned entry

- Signed by "EMAILADDRESS=root@guardianproject.info, CN=guardianproject.info, O=Guardian Project, OU=FDroid Repo, L=New York, ST=New York, C=US"
    Digest algorithm: SHA1 (disabled)
    Signature algorithm: SHA1withRSA (disabled), 4096-bit key

WARNING: The jar will be treated as unsigned, because it is signed with a weak algorithm that is now disabled by the security property:

  jdk.jar.disabledAlgorithms=MD2, MD5, RSA keySize < 1024, DSA keySize < 1024, SHA1 denyAfter 2019-01-01, include jdk.disabled.namedCurves
2022-11-14 17:59:56 +01:00
Hans-Christoph Steiner
890513fcba index: standardized AntiFeature variable names
This is the only occurance of "anti_features".
2022-11-01 09:54:06 +01:00