Compare commits

...

93 Commits

Author SHA1 Message Date
Frédéric Guillot 4d7d554df8 docs: update release notes for version 2.2.11 2025-07-26 13:13:53 -07:00
Frédéric Guillot 66b269e6cd feat(readability): avoid removing elements with content class 2025-07-25 19:59:47 -07:00
Frédéric Guillot 54abd0a736 fix(parser): handle feeds with leading whitespace that exceeds buffer size 2025-07-23 21:06:15 -07:00
Frédéric Guillot 5eab4753e8 refactor(googlereader): rename/unexport response types and functions 2025-07-23 20:36:04 -07:00
Frédéric Guillot bf466425db ci: update commit linter to allow default Git revert message 2025-07-23 20:03:04 -07:00
jvoisin a62b97bddd refactor(readability): get rid of getClassWeight
Its naming was confusing, and its code simple enough that it could be inlined.
2025-07-23 19:55:47 -07:00
jvoisin 1de9cf4241 perf(readability): simplify removeUnlikelyCandidates
- Use an iterator instead of generating a whole slice when iterating on the selection.
- Using an iterator allows to use a for-loop construct, instead of a lambda,
  which is a bit clearer
- Do the filtering Find()'s selector, instead of in the loop, which doesn't
  matter much now that we're using an iterator, but it makes the code a bit
  more obvious/simpler, and likely reduces a bit the number of iterations.
2025-07-23 19:55:47 -07:00
jvoisin 7912b9b8fb perf(readability): avoid materializing text to count commas
There is no need to materialize the whole text content of the selection only to
count its number of commas. As we already have a getLengthOfTextContent
function that is pretty similar, this commit refactors it to make it more
generic, in the form of a map/fold(+).
2025-07-23 19:55:47 -07:00
jvoisin 2d24f5d04e refactor(readability): minor code folding 2025-07-23 19:55:47 -07:00
Frédéric Guillot 20825a92c5 Revert "perf(template): use ParseFS to directly parse the embedded template data"
This reverts commit 4336a0bd85.
2025-07-22 21:31:34 -07:00
Frédéric Guillot 1d1162327e feat(integration): use Bearer token authorization instead of cookie for Linkwarden client 2025-07-22 21:13:48 -07:00
Frédéric Guillot 202de7c787 fix(integration): rename Linkwarden endpoint label to base URL (#3568) 2025-07-22 20:49:54 -07:00
Frédéric Guillot b470b186b3 feat(makefile)!: remove unsupported platforms and stop distributing Windows binary
There is no installation wizard for Windows and running the
command line binary as-is could lead to confusion for some users.

Unit tests still run on Windows, and people can still compile from
source if interested.
2025-07-22 20:24:39 -07:00
dependabot[bot] 5877cf340a build(deps): bump github.com/go-webauthn/webauthn from 0.13.3 to 0.13.4
Bumps [github.com/go-webauthn/webauthn](https://github.com/go-webauthn/webauthn) from 0.13.3 to 0.13.4.
- [Release notes](https://github.com/go-webauthn/webauthn/releases)
- [Commits](https://github.com/go-webauthn/webauthn/compare/v0.13.3...v0.13.4)

---
updated-dependencies:
- dependency-name: github.com/go-webauthn/webauthn
  dependency-version: 0.13.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-22 17:31:06 -07:00
dependabot[bot] d417b0ff12 build(deps): bump github.com/tdewolff/minify/v2 from 2.23.9 to 2.23.10
Bumps [github.com/tdewolff/minify/v2](https://github.com/tdewolff/minify) from 2.23.9 to 2.23.10.
- [Release notes](https://github.com/tdewolff/minify/releases)
- [Commits](https://github.com/tdewolff/minify/compare/v2.23.9...v2.23.10)

---
updated-dependencies:
- dependency-name: github.com/tdewolff/minify/v2
  dependency-version: 2.23.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-22 17:27:56 -07:00
Frédéric Guillot 703f113fbd fix(storage): ensure strings are truncated to fit tsvector size limit and remain valid UTF-8 2025-07-20 13:42:58 -07:00
Frédéric Guillot f02213a168 refactor(readability): use String explicitly in debug log instead of Any 2025-07-19 10:58:49 -07:00
Frédéric Guillot 410b43a787 chore(contrib): update PostgreSQL image from 15 to latest in docker-compose examples 2025-07-19 10:51:00 -07:00
Frédéric Guillot d9de9d1852 feat(rss): fallback to enclosure URL when entry URL is missing 2025-07-19 10:46:43 -07:00
dependabot[bot] 33d55cc4e9 build(deps): bump github.com/tdewolff/minify/v2 from 2.23.8 to 2.23.9
Bumps [github.com/tdewolff/minify/v2](https://github.com/tdewolff/minify) from 2.23.8 to 2.23.9.
- [Release notes](https://github.com/tdewolff/minify/releases)
- [Commits](https://github.com/tdewolff/minify/compare/v2.23.8...v2.23.9)

---
updated-dependencies:
- dependency-name: github.com/tdewolff/minify/v2
  dependency-version: 2.23.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-18 17:35:03 -07:00
Julien Voisin 1825320369 refactor(template): remove unused functions and reduce the complexity of truncate function
- Remove unused functions like hasKey, domain, hasPrefix and contains.
- Lower the complexity of truncate from O(n) to O(1).
2025-07-17 20:53:41 -07:00
jvoisin d80fb242db refactor(template): reduce translation-related introspection
Keys in translation maps are always strings, never anything else, so there is
no need to introspect them.
2025-07-17 20:48:36 -07:00
Julien Voisin 4336a0bd85 perf(template): use ParseFS to directly parse the embedded template data
Use ParseFS to directly parse the embedded template data, instead of manually
reading it and then using Parse.
2025-07-17 20:46:33 -07:00
Frédéric Guillot dc81725788 fix(filter): remove \r\n in rule parsing 2025-07-16 21:03:53 -07:00
Julien Voisin 86e2ce6d0b perf(readability): move transformMisusedDivsIntoParagraphs call after removeUnlikelyCandidates 2025-07-13 14:34:14 -07:00
Frédéric Guillot 4679691c94 refactor(googlereader): rename stream suffix constants for clarity 2025-07-13 14:28:25 -07:00
Julien Voisin 0d5f4a710f refactor(googlereader): unexport a lot of symbols 2025-07-13 14:16:08 -07:00
dependabot[bot] 92d2ac4f58 build(deps): bump github.com/go-webauthn/webauthn from 0.13.1 to 0.13.3
Bumps [github.com/go-webauthn/webauthn](https://github.com/go-webauthn/webauthn) from 0.13.1 to 0.13.3.
- [Release notes](https://github.com/go-webauthn/webauthn/releases)
- [Commits](https://github.com/go-webauthn/webauthn/compare/v0.13.1...v0.13.3)

---
updated-dependencies:
- dependency-name: github.com/go-webauthn/webauthn
  dependency-version: 0.13.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-11 17:09:37 -07:00
dependabot[bot] 1cfee27a50 build(deps): bump golang.org/x/image from 0.28.0 to 0.29.0
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.28.0 to 0.29.0.
- [Commits](https://github.com/golang/image/compare/v0.28.0...v0.29.0)

---
updated-dependencies:
- dependency-name: golang.org/x/image
  dependency-version: 0.29.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-10 19:32:49 -07:00
jvoisin 0e9da3a090 refactor(icon): simplify findIconURLsFromHTMLDocument
- Don't define the queries before possible early returns
- Check for the presence of the href attribute in the queries, instead of later
  on iterating on the selection
- Add two edge-cases to the tests
- Use EachIter instead of Each, if only to avoid the lambda
2025-07-10 19:32:29 -07:00
jvoisin 57bd384951 refactor(icon): unexport a bunch of symbols 2025-07-10 19:32:29 -07:00
dependabot[bot] fdbd5b08a1 build(deps): bump golang.org/x/net from 0.41.0 to 0.42.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.41.0 to 0.42.0.
- [Commits](https://github.com/golang/net/compare/v0.41.0...v0.42.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-10 19:24:08 -07:00
jvoisin f455c18c66 perf(rewrite): anchor the rewrite regex
There is no need to try to match the regexp over the whole input, having it
anchored is enough. If we feel extra-lenient, we might strip spaces in
front/tail, but I don't think it's necessary.

This commit also invert a condition to reduce the level of nested indentation,
and make a condition stricter.
2025-07-10 19:23:54 -07:00
dependabot[bot] 9dea26c923 build(deps): bump golang.org/x/crypto from 0.39.0 to 0.40.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.39.0 to 0.40.0.
- [Commits](https://github.com/golang/crypto/compare/v0.39.0...v0.40.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.40.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-10 17:33:04 -07:00
jvoisin 46adb0ffad refactor(fetcher): simplification of ExecuteRequest
Instead of doing some ciphers manipulation before instantiating the http.Transport
and then assigning them, instantiate http.Transport, and then in an if do the
manipulation. This makes the code a bit clearer, which is always nice when it
comes to cryptographic shenanigans.
2025-07-09 19:36:36 -07:00
jvoisin 61583d53d5 refactor(config): simplify SortedOptions
Make use of the slices and maps packages instead of doing things by hand,
and pre-allocated sortedOptions.
2025-07-09 19:29:53 -07:00
jvoisin 7c42e777ec refactor(config): minor improvements of the config parser
- Surface the faulty line number when trying to parse it
- Use strings.Cut instead of strings.SplitN
- Use strings.TrimSuffix instead of an if
- Simplify parseStringList and make its code more compact
2025-07-09 19:28:02 -07:00
dependabot[bot] 335dffbb75 build(deps): bump golang.org/x/term from 0.32.0 to 0.33.0
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.32.0 to 0.33.0.
- [Commits](https://github.com/golang/term/compare/v0.32.0...v0.33.0)

---
updated-dependencies:
- dependency-name: golang.org/x/term
  dependency-version: 0.33.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-09 19:18:08 -07:00
Qeynos f0cdfb33dd feat(locale): update Chinese translations 2025-07-09 19:14:40 -07:00
jvoisin 13ef89f785 fix(storage): don't truncate title/content fields
As found out by @fguillot in https://github.com/miniflux/v2/pull/3461#issuecomment-3047204354
2025-07-08 19:36:18 -07:00
jvoisin 32fbb4e882 fix(ui): improve margins consistency wrt. header/footer 2025-07-08 19:30:13 -07:00
Julien Voisin 135ce1d546 feat(cookie): use SameSiteStrictMode when not using OAuth2/OIDC 2025-07-08 19:20:24 -07:00
Cthulhux abed7b11ce feat(locale): update German translation
Missed two weird translations and one typo
2025-07-08 19:17:28 -07:00
Frédéric Guillot 2e26f5ca75 test(reader): ensure consistent tags parsing across feed formats 2025-07-07 20:07:35 -07:00
jvoisin d6d18a2d61 perf(reader): shrink the json detection buffer
There is no need to allocate half a kilobyte of memory only check that a buffer
starts with a bunch of spaces and a `{`, 32b should be more than enough. Also,
no need to allocate it on the heap, having it on the stack works perfectly.
2025-07-07 19:21:59 -07:00
Frédéric Guillot 63891501e5 refactor(model): add test coverage and simplify ProxifyEnclosureURL 2025-07-07 18:41:44 -07:00
dependabot[bot] 7107ff985f build(deps): bump github.com/go-webauthn/webauthn from 0.13.0 to 0.13.1
Bumps [github.com/go-webauthn/webauthn](https://github.com/go-webauthn/webauthn) from 0.13.0 to 0.13.1.
- [Release notes](https://github.com/go-webauthn/webauthn/releases)
- [Commits](https://github.com/go-webauthn/webauthn/compare/v0.13.0...v0.13.1)

---
updated-dependencies:
- dependency-name: github.com/go-webauthn/webauthn
  dependency-version: 0.13.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-07 18:39:51 -07:00
jvoisin 50d5cb96c8 refactor(enclosure): simplify model/enclosure.go
- Reuse isAudio and isVideo instead of re-implementing them
- In IsImage, return early is the mimetype is enough, instead of systematically
  lowercasing the url as well.
- Extract the common parts of the two ProxifyAbsoluteURL implementations into a
  private function, make the code smaller and clearer.
- Fix a logic error where `A && B || C` was used instead of `A && (B || C)
2025-07-07 17:42:44 -07:00
Frédéric Guillot f860daef7f test(locale): increase test coverage to 100% 2025-07-07 17:39:56 -07:00
Frédéric Guillot e7b98afdbe refactor(subscription): avoid using Sprintf to construct Youtube playlist feed URL 2025-07-07 17:08:47 -07:00
Frédéric Guillot 2cfeefc8d2 test(processor): increase test coverage for parseISO8601Duration 2025-07-07 17:01:10 -07:00
jvoisin b48e6472f5 refactor(processor): parse ~ISO8601 in a proper way
Instead of using an ugly (and incomplete) regex, let's use a simple for-loop to
parse ISO8601 dates, and make it explicit that we're only supporting a subset
of the spec, as we only care about youtube video durations.
2025-07-07 16:28:58 -07:00
jvoisin 24043ece07 refactor(config): unexport some symbols 2025-07-07 16:01:21 -07:00
jvoisin a09129d220 refactor(proxyrotator): simplify mutex handling
There is no need to use a mutex to check the length of the proxies list,
as it's read-only during the whole lifetime of a ProxyRotator structure.
Moreover, it's a bit clearer to explicitly wrap the 2 lines of mutex-needing
operations between a Lock/Unlock instead of using a defer.
2025-07-07 15:52:16 -07:00
jvoisin f864a2ed70 refactor(mediaproxy): simplify shouldProxy
The original function was non-trivial to understand, as `!A && (B || !C)` isn't
easily grokable by humans.
2025-07-07 15:50:13 -07:00
jvoisin 052e8dd0aa refactor(config): remove two unused constants 2025-07-07 15:45:20 -07:00
jvoisin 7a394b0bf8 refactor(subscription): replace a regex with strings.CutPrefix 2025-07-07 15:44:45 -07:00
jvoisin dcfe0a7d94 refactor(locale): simplify pluralForm
Instead of having a switch-case returning a function to be executed, it's
simpler/faster to have a single function containing a switch-case. It also
allows to group languages with identical plural form in a single
implementation, and remove the "default" guard value, as switch-case already
have a `default:` case.
2025-07-07 15:30:41 -07:00
jvoisin 33c648825f refactor(locale): make Printf's code structure similar to Print's
And also change the order of the cases in the Plural function, to make it
explicit that []string shouldn't match []any.
2025-07-07 15:30:41 -07:00
jvoisin 78c7f66df7 refactor(locale): remove a call to fmt.Sprintf 2025-07-07 15:30:41 -07:00
jvoisin f2b805850c refactor(locale): remove an unused function 2025-07-07 15:30:41 -07:00
jvoisin 915b7b3cf7 refactor(locale): unexport a symbol 2025-07-07 15:30:41 -07:00
jvoisin 8e86004936 refactor(locale): use any instead of interface{} 2025-07-07 15:30:41 -07:00
Julien Voisin a8b4e88742 perf(sanitizer): improve the performances of the sanitizer (#3497)
- Grow the underlying buffer of SanitizeHTML's strings.Builder to 3/4 of the
  raw HTML from the start, to reduce the amount of iterative allocations. This
  number is a complete guesstimation, but it sounds reasonable to me.
- Add a `absoluteURLParsedBase` function to avoid parsing baseURL over and over.
2025-07-07 15:21:13 -07:00
jvoisin 15e4c3a374 refactor(database): get rid of the sqlite tentative 2025-07-02 17:18:03 -07:00
jvoisin 69a74c4abf refactor(readability): minor clean up
Remove a now-useless regex and its associated test.
2025-07-02 16:50:49 -07:00
jvoisin 766d4ab834 refactor(readability): make use of getSelectionLength 2025-07-02 16:47:27 -07:00
Frédéric Guillot cb617ff6e0 test(sanitizer): enhance tests for image width and height attributes 2025-07-01 20:52:45 -07:00
Frédéric Guillot 8c3f280f32 test(readability): add test case for ExtractContent with broken reader 2025-07-01 20:14:52 -07:00
jvoisin 8a98926674 refactor(readability): add a getSelectionLength function
When we're only interested in the length of contained Text, there is no need to
materialize it fully to then call len() on the result: we can simply iterate
over the text element and sum their length instead.
2025-07-01 19:52:53 -07:00
jvoisin 435a950d64 refactor(sanitizer): minor refactorization
Use a proper switch-case instead of a bunch of if.
2025-07-01 19:48:55 -07:00
jvoisin 89c32d518d perf(readability): significantly improve transformMisusedDivsIntoParagraphs 2025-07-01 19:44:58 -07:00
jvoisin 2f7b2e7375 perf(readability): improve getLinkDensity
- There is no need to materialize all the content of a given Node when we can
  simply compute its length directly, saving a lot of memory, on the order of
  several megabytes on my instance, with peaks at a couple of dozen.
- One might object to the usage of a recursive construct, but this is a direct
  port of goquery's Text method, so this change doesn't make anything worse.
- The computation of linkLength can be similarly computed, but this can go in
  another commit, as it's a bit trickier, since we need to get the length of
  every Node that has a `a` Node as parent, without iterating on the whole
  parent chain every time.
2025-07-01 19:40:47 -07:00
Frédéric Guillot 6eeccae7cd test(readability): increase test coverage 2025-06-30 21:29:07 -07:00
jvoisin 99c5bcdb01 perf(storage): truncate strings on go's side instead of pgsql's
There is no need to send the whole title and content to have them truncated on
postgresql's side when we can do this client-side. This should save some
memory on the database's side, as well as some bandwidth
if it's located on another server. And it makes the SQL queries a tad more
readable as well.
2025-06-30 19:45:45 -07:00
jvoisin aed99e65c1 perf(readability): improve getClassWeight speed
Before

```console
$ go test -bench=.
goos: linux
goarch: arm64
pkg: miniflux.app/v2/internal/reader/readability
BenchmarkExtractContent-8   	     34	 86102474 ns/op
BenchmarkGetWeight-8        	  10573	    103045 ns/op
PASS
ok  	miniflux.app/v2/internal/reader/readability	5.409s
```

After

```console
$ go test -bench=.
goos: linux
goarch: arm64
pkg: miniflux.app/v2/internal/reader/readability
BenchmarkExtractContent-8   	     56	 83130924 ns/op
BenchmarkGetWeight-8        	 246541	     5241 ns/op
PASS
ok  	miniflux.app/v2/internal/reader/readability	6.026s
```

This should make ProcessFeedEntries marginally faster, while saving
some memory.
2025-06-30 19:28:20 -07:00
jvoisin d1a3f98df9 perf(fetcher): save 8 bytes in the RequestBuilder struct
before:

```
  // request_builder.go:25 | Size: 64 (Optimal: 56)
  type RequestBuilder struct {
    headers          http.Header                 ■ ■ ■ ■ ■ ■ ■ ■
    clientProxyURL   *url.URL                    ■ ■ ■ ■ ■ ■ ■ ■
    useClientProxy   bool                        ■ □ □ □ □ □ □ □
    clientTimeout    int                         ■ ■ ■ ■ ■ ■ ■ ■
    withoutRedirects bool                        ■
    ignoreTLSErrors  bool                          ■
    disableHTTP2     bool                            ■ □ □ □ □ □
    proxyRotator     *proxyrotator.ProxyRotator  ■ ■ ■ ■ ■ ■ ■ ■
    feedProxyURL     string                      ■ ■ ■ ■ ■ ■ ■ ■
                                                 ■ ■ ■ ■ ■ ■ ■ ■
  }
```

after:

```
  // request_builder.go:25 | Size: 56
  type RequestBuilder struct {
    headers          http.Header                 ■ ■ ■ ■ ■ ■ ■ ■
    clientProxyURL   *url.URL                    ■ ■ ■ ■ ■ ■ ■ ■
    clientTimeout    int                         ■ ■ ■ ■ ■ ■ ■ ■
    useClientProxy   bool                        ■
    withoutRedirects bool                          ■
    ignoreTLSErrors  bool                            ■
    disableHTTP2     bool                              ■ □ □ □ □
    proxyRotator     *proxyrotator.ProxyRotator  ■ ■ ■ ■ ■ ■ ■ ■
    feedProxyURL     string                      ■ ■ ■ ■ ■ ■ ■ ■
                                                 ■ ■ ■ ■ ■ ■ ■ ■
  }
```
2025-06-29 16:10:35 -07:00
jvoisin 112494bb66 perf(feed): save 16 bytes in the Feed struct
before:

```
  // feed.go:25 | Size: 560 (Optimal: 544)
  type Feed struct {
    ID                          int64      ■ ■ ■ ■ ■ ■ ■ ■
    UserID                      int64      ■ ■ ■ ■ ■ ■ ■ ■
    FeedURL                     string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    SiteURL                     string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Title                       string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Description                 string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    CheckedAt                   time.Time  ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    NextCheckAt                 time.Time  ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    EtagHeader                  string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    LastModifiedHeader          string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    ParsingErrorMsg             string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    ParsingErrorCount           int        ■ ■ ■ ■ ■ ■ ■ ■
    ScraperRules                string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    RewriteRules                string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Crawler                     bool       ■ □ □ □ □ □ □ □
    BlocklistRules              string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    KeeplistRules               string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    BlockFilterEntryRules       string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    KeepFilterEntryRules        string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    UrlRewriteRules             string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    UserAgent                   string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Cookie                      string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Username                    string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Password                    string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Disabled                    bool       ■
    NoMediaPlayer               bool         ■
    IgnoreHTTPCache             bool           ■
    AllowSelfSignedCertificates bool             ■
    FetchViaProxy               bool               ■
    HideGlobally                bool                 ■
    DisableHTTP2                bool                   ■ □
    AppriseServiceURLs          string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    WebhookURL                  string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    NtfyEnabled                 bool       ■ □ □ □ □ □ □ □
    NtfyPriority                int        ■ ■ ■ ■ ■ ■ ■ ■
    NtfyTopic                   string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    PushoverEnabled             bool       ■ □ □ □ □ □ □ □
    PushoverPriority            int        ■ ■ ■ ■ ■ ■ ■ ■
    ProxyURL                    string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Category                    *Category  ■ ■ ■ ■ ■ ■ ■ ■
    Icon                        *FeedIcon  ■ ■ ■ ■ ■ ■ ■ ■
    Entries                     Entries    ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    TTL                         int        ■ ■ ■ ■ ■ ■ ■ ■
    IconURL                     string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    UnreadCount                 int        ■ ■ ■ ■ ■ ■ ■ ■
    ReadCount                   int        ■ ■ ■ ■ ■ ■ ■ ■
    NumberOfVisibleEntries      int        ■ ■ ■ ■ ■ ■ ■ ■
  }
```

after:

```
// feed.go:25 | Size: 544
  type Feed struct {
    ID                          int64      ■ ■ ■ ■ ■ ■ ■ ■
    UserID                      int64      ■ ■ ■ ■ ■ ■ ■ ■
    FeedURL                     string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    SiteURL                     string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Title                       string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Description                 string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    CheckedAt                   time.Time  ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    NextCheckAt                 time.Time  ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    EtagHeader                  string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    LastModifiedHeader          string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    ParsingErrorMsg             string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    ParsingErrorCount           int        ■ ■ ■ ■ ■ ■ ■ ■
    ScraperRules                string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    RewriteRules                string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    BlocklistRules              string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    KeeplistRules               string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    BlockFilterEntryRules       string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    KeepFilterEntryRules        string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    UrlRewriteRules             string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    UserAgent                   string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Cookie                      string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Username                    string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Password                    string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Disabled                    bool       ■
    NoMediaPlayer               bool         ■
    IgnoreHTTPCache             bool           ■
    AllowSelfSignedCertificates bool             ■
    FetchViaProxy               bool               ■
    HideGlobally                bool                 ■
    DisableHTTP2                bool                   ■
    PushoverEnabled             bool                     ■
    NtfyEnabled                 bool       ■
    Crawler                     bool         ■ □ □ □ □ □ □
    AppriseServiceURLs          string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    WebhookURL                  string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    NtfyPriority                int        ■ ■ ■ ■ ■ ■ ■ ■
    NtfyTopic                   string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    PushoverPriority            int        ■ ■ ■ ■ ■ ■ ■ ■
    ProxyURL                    string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    Category                    *Category  ■ ■ ■ ■ ■ ■ ■ ■
    Icon                        *FeedIcon  ■ ■ ■ ■ ■ ■ ■ ■
    Entries                     Entries    ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    TTL                         int        ■ ■ ■ ■ ■ ■ ■ ■
    IconURL                     string     ■ ■ ■ ■ ■ ■ ■ ■
                                           ■ ■ ■ ■ ■ ■ ■ ■
    UnreadCount                 int        ■ ■ ■ ■ ■ ■ ■ ■
    ReadCount                   int        ■ ■ ■ ■ ■ ■ ■ ■
    NumberOfVisibleEntries      int        ■ ■ ■ ■ ■ ■ ■ ■
  }
```
2025-06-29 16:10:35 -07:00
jvoisin 9f7ecdb75a perf(model): save 16 bytes in the FeedCreationRequest struct
before:

```
  // feed.go:154 | Size: 240 (Optimal: 224)
  type FeedCreationRequest struct {
    FeedURL                     string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    CategoryID                  int64   ■ ■ ■ ■ ■ ■ ■ ■
    UserAgent                   string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    Cookie                      string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    Username                    string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    Password                    string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    Crawler                     bool    ■
    Disabled                    bool      ■
    NoMediaPlayer               bool        ■
    IgnoreHTTPCache             bool          ■
    AllowSelfSignedCertificates bool            ■
    FetchViaProxy               bool              ■ □ □
    UrlRewriteRules             string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    DisableHTTP2                bool    ■ □ □ □ □ □ □ □
    ScraperRules                string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    RewriteRules                string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    BlocklistRules              string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    KeeplistRules               string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    BlockFilterEntryRules       string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    KeepFilterEntryRules        string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    HideGlobally                bool    ■ □ □ □ □ □ □ □
    ProxyURL                    string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
  }
```

after:

```
  // feed.go:154 | Size: 224
  type FeedCreationRequest struct {
    FeedURL                     string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    CategoryID                  int64   ■ ■ ■ ■ ■ ■ ■ ■
    UserAgent                   string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    Cookie                      string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    Username                    string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    Password                    string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    Crawler                     bool    ■
    Disabled                    bool      ■
    NoMediaPlayer               bool        ■
    IgnoreHTTPCache             bool          ■
    AllowSelfSignedCertificates bool            ■
    FetchViaProxy               bool              ■
    HideGlobally                bool                ■
    DisableHTTP2                bool                  ■
    ScraperRules                string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    RewriteRules                string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    BlocklistRules              string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    KeeplistRules               string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    BlockFilterEntryRules       string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    KeepFilterEntryRules        string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    UrlRewriteRules             string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
    ProxyURL                    string  ■ ■ ■ ■ ■ ■ ■ ■
                                        ■ ■ ■ ■ ■ ■ ■ ■
  }
```
2025-06-29 16:10:35 -07:00
jvoisin 4e1f836266 refactor(readability): simplify a bit getArticle
- Use a proper division instead of multiplying by a float.
- Extract a condition in the parent scope
- Use an else-if construct instead of a simple if
2025-06-29 16:06:34 -07:00
Frédéric Guillot a68de4ee6a test(readability): add tests for getArticle function 2025-06-29 16:03:17 -07:00
jvoisin c064891314 perf(readability): Simplify removeUnlikelyCandidates
- Use an array of strings instead of a regex, like done in ef13756b1a7a7ba30fd34174a5367381fd8b4849
- Extract the `shouldRemove` function from `removeUnlikelyCandidates`, as there
  is no reason to have it there instead of being a proper standalone function.
- Improve a condition, where the goquery selection would have its `id`
  attribute left unchecked if a `class` one was present, regardless of if
  `class` was a candidate to removal or not.
- Add some comments
2025-06-29 15:31:01 -07:00
Frédéric Guillot 5129f53d58 test(readability): add tests for removeUnlikelyCandidates function 2025-06-29 15:23:56 -07:00
Frédéric Guillot e60f0fd142 test(readability): add tests for getClassWeight function 2025-06-29 13:24:06 -07:00
Julien Voisin 2b26a345cd perf(processor): minify content even further
There is no need to keep comments (conditionals or not, as IE isn't a thing
anymore), nor default attribute values.
2025-06-29 12:55:34 -07:00
Frédéric Guillot 3de31a1a4d test(processor): add more unit tests for minifyContent function 2025-06-29 12:53:23 -07:00
jvoisin 560be66147 refactor(misc): Use proper slog.XXX instead of slog.Any
This has close to no impact for now, as our slog.Debug/Info/... are leaking
their parameters to the heap, but using proper typing instead of Any allows
to skip some reflection-based computation, making things marginally faster,
and removing the corresponding heap leak.
2025-06-29 12:30:17 -07:00
Ingmar Stein fcf86e33b9 feat: TLS support for Unix socket listeners
This change enables Miniflux to serve TLS over Unix domain sockets.

If `CERT_FILE` and `KEY_FILE` are configured, Unix socket listeners
specified via `LISTEN_ADDR` will now automatically start with TLS enabled,
using the provided certificates. This uses the existing `http.Server.ServeTLS`
method.

If no certificates are provided, Unix socket listeners will continue to
operate as plain, non-TLS sockets.
2025-06-24 21:25:55 -07:00
dependabot[bot] 113f6b8982 build(deps): bump github.com/andybalholm/brotli from 1.1.1 to 1.2.0
Bumps [github.com/andybalholm/brotli](https://github.com/andybalholm/brotli) from 1.1.1 to 1.2.0.
- [Commits](https://github.com/andybalholm/brotli/compare/v1.1.1...v1.2.0)

---
updated-dependencies:
- dependency-name: github.com/andybalholm/brotli
  dependency-version: 1.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-24 19:52:21 -07:00
jvoisin cbdcf1a56c Revert "perf(storage): take advantage of entries_feed_id_hash_key in updateEntry"
This reverts commit 6527c04307.
2025-06-24 19:51:21 -07:00
Frédéric Guillot 95eb6c1230 chore(docker): update golang base image to alpine 3.22 2025-06-23 19:11:03 -07:00
jvoisin 643b89ec89 perf(storage): take advantage of entries_feed_id_hash_key in updateEntry 2025-06-23 17:59:17 -07:00
Frédéric Guillot 84ebf1a033 docs(manpage): update LISTEN_ADDR description 2025-06-23 17:51:02 -07:00
99 changed files with 5410 additions and 1617 deletions
+10 -20
View File
@@ -4,8 +4,10 @@ import sys
import argparse
from typing import Match
# Conventional commit pattern
CONVENTIONAL_COMMIT_PATTERN: str = r"^(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test)(\([a-z0-9-]+\))?!?: .{1,100}"
# Conventional commit pattern (including Git revert messages)
CONVENTIONAL_COMMIT_PATTERN: str = (
r"^((build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test)(\([a-z0-9-]+\))?!?: .{1,100}|Revert .+)"
)
def get_commit_message(commit_hash: str) -> str:
@@ -23,9 +25,7 @@ def get_commit_message(commit_hash: str) -> str:
sys.exit(1)
def check_commit_message(
message: str, pattern: str = CONVENTIONAL_COMMIT_PATTERN
) -> bool:
def check_commit_message(message: str, pattern: str = CONVENTIONAL_COMMIT_PATTERN) -> bool:
"""Check if commit message follows conventional commit format."""
first_line: str = message.split("\n")[0]
match: Match[str] | None = re.match(pattern, first_line)
@@ -50,9 +50,7 @@ def check_commit_range(base_ref: str, head_ref: str) -> list[dict[str, str]]:
for commit_hash in commit_hashes:
message: str = get_commit_message(commit_hash)
if not check_commit_message(message):
non_compliant.append(
{"hash": commit_hash, "message": message.split("\n")[0]}
)
non_compliant.append({"hash": commit_hash, "message": message.split("\n")[0]})
return non_compliant
except subprocess.CalledProcessError as e:
@@ -61,15 +59,9 @@ def check_commit_range(base_ref: str, head_ref: str) -> list[dict[str, str]]:
def main() -> None:
parser: argparse.ArgumentParser = argparse.ArgumentParser(
description="Check conventional commit compliance"
)
parser.add_argument(
"--base", required=True, help="Base ref (starting commit, exclusive)"
)
parser.add_argument(
"--head", required=True, help="Head ref (ending commit, inclusive)"
)
parser: argparse.ArgumentParser = argparse.ArgumentParser(description="Check conventional commit compliance")
parser.add_argument("--base", required=True, help="Base ref (starting commit, exclusive)")
parser.add_argument("--head", required=True, help="Head ref (ending commit, inclusive)")
args: argparse.Namespace = parser.parse_args()
non_compliant: list[dict[str, str]] = check_commit_range(args.base, args.head)
@@ -80,9 +72,7 @@ def main() -> None:
print(f"- {commit['hash'][:8]}: {commit['message']}")
print("\nPlease ensure your commit messages follow the format:")
print("type(scope): subject")
print(
"\nWhere type is one of: build, chore, ci, docs, feat, fix, perf, refactor, revert, style, test"
)
print("\nWhere type is one of: build, chore, ci, docs, feat, fix, perf, refactor, revert, style, test")
sys.exit(1)
else:
print("All commits follow the conventional commit format!")
+59
View File
@@ -1,3 +1,62 @@
Version 2.2.11 (July 26, 2025)
------------------------------
### ✨ New Features
* **TLS support for Unix sockets**: Miniflux can now serve TLS over Unix domain sockets using `CERT_FILE` and `KEY_FILE` ([#fcf86e3](https://github.com/miniflux/v2/commit/fcf86e3)).
* **RSS fallback**: If a feed entry has no URL, Miniflux now uses the enclosure URL as a fallback ([#d9de9d1](https://github.com/miniflux/v2/commit/d9de9d1)).
* **Bearer token for Linkwarden**: The Linkwarden integration now uses Bearer token authorization instead of cookies ([#1d11623](https://github.com/miniflux/v2/commit/1d11623)).
* **Cookie policy improvement**: `SameSiteStrictMode` is enforced for cookies when OAuth2/OIDC is not used ([#135ce1d](https://github.com/miniflux/v2/commit/135ce1d)).
* **Readability engine**: Avoid removing elements with the `content` class during readability parsing ([#66b269e](https://github.com/miniflux/v2/commit/66b269e)).
### 🛠️ Improvements
* **Massive readability engine refactoring** and performance optimizations:
* Improved performance of `getClassWeight`, `getLinkDensity`, and `transformMisusedDivsIntoParagraphs`.
* Simplified and optimized internal logic of `removeUnlikelyCandidates`, `getSelectionLength`, and `getArticle`.
* Reduced memory allocation in sanitizer and readability components.
* **Storage optimization**: Strings are now truncated on the Go side to respect `tsvector` limits, reducing DB load and ensuring valid UTF-8 ([#703f113](https://github.com/miniflux/v2/commit/703f113)).
* **Simplified and clarified internal code structure**:
* Major cleanup and size optimization of internal structs (`Feed`, `FeedCreationRequest`, etc.).
* Reduced memory use and improved CPU cache locality.
* Numerous refactors across `config`, `template`, `locale`, `subscription`, and `fetcher` modules.
### 🐛 Bug Fixes
* Fixed an issue with feeds having excessive leading whitespace causing parser buffer issues ([#54abd0a](https://github.com/miniflux/v2/commit/54abd0a)).
* Properly preserve UTF-8 when truncating strings for full-text search ([#703f113](https://github.com/miniflux/v2/commit/703f113)).
* Fixed logic error in enclosure type detection ([#50d5cb9](https://github.com/miniflux/v2/commit/50d5cb9)).
* Fixed incorrect filter rule parsing of Windows-style newlines ([#dc81725](https://github.com/miniflux/v2/commit/dc81725)).
* Fixed a panic in `startAutoCertTLSServer` function when using Let's Encrypt automatic certificates ([#f7a6b02](https://github.com/miniflux/v2/commit/f7a6b02))
* Improved UI spacing consistency around header/footer ([#32fbb4e](https://github.com/miniflux/v2/commit/32fbb4e)).
### ⚠️ Breaking Changes
* **Windows binary no longer distributed**: Windows is no longer a supported platform for binary distribution. Users must build from source if needed ([#b470b18](https://github.com/miniflux/v2/commit/b470b18)).
### 🧪 Tests & CI
* Test coverage significantly increased for modules like `readability`, `sanitizer`, `processor`, `locale`, and `storage`.
* Commit linter updated to support new Git revert message format.
### 🐘 Docker & Environment
* Base Docker image updated to Alpine 3.22.
* PostgreSQL Docker example updated to use the latest version.
### 🌐 Localization
* Updated Chinese and German translations.
### 🔒 Dependency Updates
* Bumped `github.com/go-webauthn/webauthn` to `0.13.4`
* Bumped `github.com/tdewolff/minify/v2` to `2.23.10`
* Bumped `golang.org/x/*` modules: `image`, `net`, `term`, `crypto`
* Bumped `github.com/andybalholm/brotli` to `1.2.0`
Version 2.2.10 (June 23, 2025)
------------------------------
+2 -31
View File
@@ -22,16 +22,12 @@ export PGPASSWORD := postgres
darwin-amd64 \
darwin-arm64 \
freebsd-amd64 \
freebsd-x86 \
openbsd-amd64 \
openbsd-x86 \
netbsd-x86 \
netbsd-amd64 \
windows-amd64 \
windows-x86 \
build \
run \
clean \
add-string \
test \
lint \
integration-test \
@@ -85,30 +81,7 @@ openbsd-amd64:
@ GOOS=openbsd GOARCH=amd64 go build -ldflags=$(LD_FLAGS) -o $(APP)-$@ main.go
@ sha256sum $(APP)-$@ > $(APP)-$@.sha256
windows-amd64:
@ GOOS=windows GOARCH=amd64 go build -ldflags=$(LD_FLAGS) -o $(APP)-$@.exe main.go
@ sha256sum $(APP)-$@.exe > $(APP)-$@.exe.sha256
build: linux-amd64 linux-arm64 linux-armv7 linux-armv6 linux-armv5 darwin-amd64 darwin-arm64 freebsd-amd64 openbsd-amd64 windows-amd64
# NOTE: unsupported targets
netbsd-amd64:
@ CGO_ENABLED=0 GOOS=netbsd GOARCH=amd64 go build -ldflags=$(LD_FLAGS) -o $(APP)-$@ main.go
linux-x86:
@ CGO_ENABLED=0 GOOS=linux GOARCH=386 go build -ldflags=$(LD_FLAGS) -o $(APP)-$@ main.go
freebsd-x86:
@ CGO_ENABLED=0 GOOS=freebsd GOARCH=386 go build -ldflags=$(LD_FLAGS) -o $(APP)-$@ main.go
netbsd-x86:
@ CGO_ENABLED=0 GOOS=netbsd GOARCH=386 go build -ldflags=$(LD_FLAGS) -o $(APP)-$@ main.go
openbsd-x86:
@ GOOS=openbsd GOARCH=386 go build -ldflags=$(LD_FLAGS) -o $(APP)-$@ main.go
windows-x86:
@ GOOS=windows GOARCH=386 go build -ldflags=$(LD_FLAGS) -o $(APP)-$@.exe main.go
build: linux-amd64 linux-arm64 linux-armv7 linux-armv6 linux-armv5 darwin-amd64 darwin-arm64 freebsd-amd64 openbsd-amd64
run:
@ LOG_DATE_TIME=1 LOG_LEVEL=debug RUN_MIGRATIONS=1 CREATE_ADMIN=1 ADMIN_USERNAME=admin ADMIN_PASSWORD=test123 go run main.go
@@ -116,7 +89,6 @@ run:
clean:
@ rm -f $(APP)-* $(APP) $(APP)*.rpm $(APP)*.deb $(APP)*.exe $(APP)*.sha256
.PHONY: add-string
add-string:
cd internal/locale/translations && \
for file in *.json; do \
@@ -125,7 +97,6 @@ add-string:
mv tmp "$$file"; \
done
test:
go test -cover -race -count=1 ./...
+1 -1
View File
@@ -19,7 +19,7 @@ services:
# healthcheck:
# test: ["CMD", "/usr/bin/miniflux", "-healthcheck", "auto"]
db:
image: postgres:15
image: postgres:latest
container_name: postgres
environment:
- POSTGRES_USER=miniflux
+1 -1
View File
@@ -25,7 +25,7 @@ services:
- ADMIN_PASSWORD=test123
- BASE_URL=https://miniflux.example.org
db:
image: postgres:15
image: postgres:latest
container_name: postgres
environment:
- POSTGRES_USER=miniflux
+1 -1
View File
@@ -37,7 +37,7 @@ services:
- "traefik.http.routers.miniflux.entrypoints=websecure"
- "traefik.http.routers.miniflux.tls.certresolver=myresolver"
db:
image: postgres:15
image: postgres:latest
container_name: postgres
environment:
- POSTGRES_USER=miniflux
+12 -13
View File
@@ -4,24 +4,23 @@ module miniflux.app/v2
require (
github.com/PuerkitoBio/goquery v1.10.3
github.com/andybalholm/brotli v1.1.1
github.com/andybalholm/brotli v1.2.0
github.com/coreos/go-oidc/v3 v3.14.1
github.com/go-webauthn/webauthn v0.13.0
github.com/go-webauthn/webauthn v0.13.4
github.com/gorilla/mux v1.8.1
github.com/lib/pq v1.10.9
github.com/mattn/go-sqlite3 v1.14.28
github.com/prometheus/client_golang v1.22.0
github.com/tdewolff/minify/v2 v2.23.8
golang.org/x/crypto v0.39.0
golang.org/x/image v0.28.0
golang.org/x/net v0.41.0
github.com/tdewolff/minify/v2 v2.23.10
golang.org/x/crypto v0.40.0
golang.org/x/image v0.29.0
golang.org/x/net v0.42.0
golang.org/x/oauth2 v0.30.0
golang.org/x/term v0.32.0
golang.org/x/term v0.33.0
)
require (
github.com/go-webauthn/x v0.1.21 // indirect
github.com/golang-jwt/jwt/v5 v5.2.2 // indirect
github.com/go-webauthn/x v0.1.23 // indirect
github.com/golang-jwt/jwt/v5 v5.2.3 // indirect
github.com/google/go-tpm v0.9.5 // indirect
)
@@ -29,7 +28,7 @@ require (
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/fxamacker/cbor/v2 v2.8.0 // indirect
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
github.com/go-jose/go-jose/v4 v4.0.5 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
@@ -39,8 +38,8 @@ require (
github.com/prometheus/procfs v0.15.1 // indirect
github.com/tdewolff/parse/v2 v2.8.1 // indirect
github.com/x448/float16 v0.8.4 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/text v0.26.0 // indirect
golang.org/x/sys v0.34.0 // indirect
golang.org/x/text v0.27.0 // indirect
google.golang.org/protobuf v1.36.5 // indirect
)
+24 -26
View File
@@ -1,7 +1,7 @@
github.com/PuerkitoBio/goquery v1.10.3 h1:pFYcNSqHxBD06Fpj/KsbStFRsgRATgnf3LeXiUkhzPo=
github.com/PuerkitoBio/goquery v1.10.3/go.mod h1:tMUX0zDMHXYlAQk6p35XxQMqMweEKB7iK7iLNd4RH4Y=
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/andybalholm/cascadia v1.3.3 h1:AG2YHrzJIm4BZ19iwJ/DAua6Btl3IwJX+VI4kktS1LM=
github.com/andybalholm/cascadia v1.3.3/go.mod h1:xNd9bqTn98Ln4DwST8/nG+H0yuB8Hmgu1YHNnWw0GeA=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
@@ -12,16 +12,16 @@ github.com/coreos/go-oidc/v3 v3.14.1 h1:9ePWwfdwC4QKRlCXsJGou56adA/owXczOzwKdOum
github.com/coreos/go-oidc/v3 v3.14.1/go.mod h1:HaZ3szPaZ0e4r6ebqvsLWlk2Tn+aejfmrfah6hnSYEU=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fxamacker/cbor/v2 v2.8.0 h1:fFtUGXUzXPHTIUdne5+zzMPTfffl3RD5qYnkY40vtxU=
github.com/fxamacker/cbor/v2 v2.8.0/go.mod h1:vM4b+DJCtHn+zz7h3FFp/hDAI9WNWCsZj23V5ytsSxQ=
github.com/fxamacker/cbor/v2 v2.9.0 h1:NpKPmjDBgUfBms6tr6JZkTHtfFGcMKsw3eGcmD/sapM=
github.com/fxamacker/cbor/v2 v2.9.0/go.mod h1:vM4b+DJCtHn+zz7h3FFp/hDAI9WNWCsZj23V5ytsSxQ=
github.com/go-jose/go-jose/v4 v4.0.5 h1:M6T8+mKZl/+fNNuFHvGIzDz7BTLQPIounk/b9dw3AaE=
github.com/go-jose/go-jose/v4 v4.0.5/go.mod h1:s3P1lRrkT8igV8D9OjyL4WRyHvjB6a4JSllnOrmmBOA=
github.com/go-webauthn/webauthn v0.13.0 h1:cJIL1/1l+22UekVhipziAaSgESJxokYkowUqAIsWs0Y=
github.com/go-webauthn/webauthn v0.13.0/go.mod h1:Oy9o2o79dbLKRPZWWgRIOdtBGAhKnDIaBp2PFkICRHs=
github.com/go-webauthn/x v0.1.21 h1:nFbckQxudvHEJn2uy1VEi713MeSpApoAv9eRqsb9AdQ=
github.com/go-webauthn/x v0.1.21/go.mod h1:sEYohtg1zL4An1TXIUIQ5csdmoO+WO0R4R2pGKaHYKA=
github.com/golang-jwt/jwt/v5 v5.2.2 h1:Rl4B7itRWVtYIHFrSNd7vhTiz9UpLdi6gZhZ3wEeDy8=
github.com/golang-jwt/jwt/v5 v5.2.2/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/go-webauthn/webauthn v0.13.4 h1:q68qusWPcqHbg9STSxBLBHnsKaLxNO0RnVKaAqMuAuQ=
github.com/go-webauthn/webauthn v0.13.4/go.mod h1:MglN6OH9ECxvhDqoq1wMoF6P6JRYDiQpC9nc5OomQmI=
github.com/go-webauthn/x v0.1.23 h1:9lEO0s+g8iTyz5Vszlg/rXTGrx3CjcD0RZQ1GPZCaxI=
github.com/go-webauthn/x v0.1.23/go.mod h1:AJd3hI7NfEp/4fI6T4CHD753u91l510lglU7/NMN6+E=
github.com/golang-jwt/jwt/v5 v5.2.3 h1:kkGXqQOBSDDWRhWNXTFpqGSCMyh/PLnqUvMGJPDJDs0=
github.com/golang-jwt/jwt/v5 v5.2.3/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
@@ -37,8 +37,6 @@ github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/mattn/go-sqlite3 v1.14.28 h1:ThEiQrnbtumT+QMknw63Befp/ce/nUPgBPMlRFEum7A=
github.com/mattn/go-sqlite3 v1.14.28/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
@@ -55,8 +53,8 @@ github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0leargg
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tdewolff/minify/v2 v2.23.8 h1:tvjHzRer46kwOfpdCBCWsDblCw3QtnLJRd61pTVkyZ8=
github.com/tdewolff/minify/v2 v2.23.8/go.mod h1:VW3ISUd3gDOZuQ/jwZr4sCzsuX+Qvsx87FDMjk6Rvno=
github.com/tdewolff/minify/v2 v2.23.10 h1:puzRCH00Im+KDf+PxuuSmJykMTVd8Pp1HzTCxVutNmI=
github.com/tdewolff/minify/v2 v2.23.10/go.mod h1:VW3ISUd3gDOZuQ/jwZr4sCzsuX+Qvsx87FDMjk6Rvno=
github.com/tdewolff/parse/v2 v2.8.1 h1:J5GSHru6o3jF1uLlEKVXkDxxcVx6yzOlIVIotK4w2po=
github.com/tdewolff/parse/v2 v2.8.1/go.mod h1:Hwlni2tiVNKyzR1o6nUs4FOF07URA+JLBLd6dlIXYqo=
github.com/tdewolff/test v1.0.11 h1:FdLbwQVHxqG16SlkGveC0JVyrJN62COWTRyUFzfbtBE=
@@ -72,10 +70,10 @@ golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliY
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.39.0 h1:SHs+kF4LP+f+p14esP5jAoDpHU8Gu/v9lFRK6IT5imM=
golang.org/x/crypto v0.39.0/go.mod h1:L+Xg3Wf6HoL4Bn4238Z6ft6KfEpN0tJGo53AAPC632U=
golang.org/x/image v0.28.0 h1:gdem5JW1OLS4FbkWgLO+7ZeFzYtL3xClb97GaUzYMFE=
golang.org/x/image v0.28.0/go.mod h1:GUJYXtnGKEUgggyzh+Vxt+AviiCcyiwpsl8iQ8MvwGY=
golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
golang.org/x/image v0.29.0 h1:HcdsyR4Gsuys/Axh0rDEmlBmB68rW1U9BUdB3UVHsas=
golang.org/x/image v0.29.0/go.mod h1:RVJROnf3SLK8d26OW91j4FrIHGbsJ8QnbEocVTOWQDA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
@@ -90,8 +88,8 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -112,8 +110,8 @@ golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@@ -123,8 +121,8 @@ golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
golang.org/x/term v0.32.0 h1:DR4lr0TjUs3epypdhTOkMmuF5CDFJ/8pOnbzMZPQ7bg=
golang.org/x/term v0.32.0/go.mod h1:uZG1FhGx848Sqfsq4/DlJr3xGGsYMu/L5GW4abiaEPQ=
golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
@@ -134,8 +132,8 @@ golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+2 -1
View File
@@ -7,6 +7,7 @@ import (
json_parser "encoding/json"
"net/http"
"miniflux.app/v2/internal/config"
"miniflux.app/v2/internal/http/request"
"miniflux.app/v2/internal/http/response/json"
"miniflux.app/v2/internal/model"
@@ -33,7 +34,7 @@ func (h *handler) getEnclosureByID(w http.ResponseWriter, r *http.Request) {
return
}
enclosure.ProxifyEnclosureURL(h.router)
enclosure.ProxifyEnclosureURL(h.router, config.Opts.MediaProxyMode(), config.Opts.MediaProxyResourceTypes())
json.OK(w, r, enclosure)
}
+2 -2
View File
@@ -10,6 +10,7 @@ import (
"strconv"
"time"
"miniflux.app/v2/internal/config"
"miniflux.app/v2/internal/http/request"
"miniflux.app/v2/internal/http/response/json"
"miniflux.app/v2/internal/integration"
@@ -34,8 +35,7 @@ func (h *handler) getEntryFromBuilder(w http.ResponseWriter, r *http.Request, b
}
entry.Content = mediaproxy.RewriteDocumentWithAbsoluteProxyURL(h.router, entry.Content)
entry.Enclosures.ProxifyEnclosureURL(h.router)
entry.Enclosures.ProxifyEnclosureURL(h.router, config.Opts.MediaProxyMode(), config.Opts.MediaProxyResourceTypes())
json.OK(w, r, entry)
}
-14
View File
@@ -7,8 +7,6 @@ import (
json_parser "encoding/json"
"errors"
"net/http"
"regexp"
"strings"
"miniflux.app/v2/internal/http/request"
"miniflux.app/v2/internal/http/response/json"
@@ -84,18 +82,6 @@ func (h *handler) updateUser(w http.ResponseWriter, r *http.Request) {
}
}
cleanEnd := regexp.MustCompile(`(?m)\r\n\s*$`)
if userModificationRequest.BlockFilterEntryRules != nil {
*userModificationRequest.BlockFilterEntryRules = cleanEnd.ReplaceAllLiteralString(*userModificationRequest.BlockFilterEntryRules, "")
// Clean carriage returns for Windows environments
*userModificationRequest.BlockFilterEntryRules = strings.ReplaceAll(*userModificationRequest.BlockFilterEntryRules, "\r\n", "\n")
}
if userModificationRequest.KeepFilterEntryRules != nil {
*userModificationRequest.KeepFilterEntryRules = cleanEnd.ReplaceAllLiteralString(*userModificationRequest.KeepFilterEntryRules, "")
// Clean carriage returns for Windows environments
*userModificationRequest.KeepFilterEntryRules = strings.ReplaceAll(*userModificationRequest.KeepFilterEntryRules, "\r\n", "\n")
}
if validationErr := validator.ValidateUserModification(h.store, originalUser.ID, &userModificationRequest); validationErr != nil {
json.BadRequest(w, r, validationErr.Error())
return
+1 -1
View File
@@ -4,4 +4,4 @@
package config // import "miniflux.app/v2/internal/config"
// Opts holds parsed configuration options.
var Opts *Options
var Opts *options
+96 -102
View File
@@ -5,8 +5,9 @@ package config // import "miniflux.app/v2/internal/config"
import (
"fmt"
"maps"
"net/url"
"sort"
"slices"
"strings"
"time"
@@ -23,8 +24,6 @@ const (
defaultHSTS = true
defaultHTTPService = true
defaultSchedulerService = true
defaultDebug = false
defaultTiming = false
defaultBaseURL = "http://localhost"
defaultRootURL = "http://localhost"
defaultBasePath = ""
@@ -95,14 +94,14 @@ const (
var defaultHTTPClientUserAgent = "Mozilla/5.0 (compatible; Miniflux/" + version.Version + "; +https://miniflux.app)"
// Option contains a key to value map of a single option. It may be used to output debug strings.
type Option struct {
// option contains a key to value map of a single option. It may be used to output debug strings.
type option struct {
Key string
Value interface{}
Value any
}
// Options contains configuration options.
type Options struct {
// options contains configuration options.
type options struct {
HTTPS bool
logFile string
logDateTime bool
@@ -184,8 +183,8 @@ type Options struct {
}
// NewOptions returns Options with default values.
func NewOptions() *Options {
return &Options{
func NewOptions() *options {
return &options{
HTTPS: defaultHTTPS,
logFile: defaultLogFile,
logDateTime: defaultLogDateTime,
@@ -264,261 +263,261 @@ func NewOptions() *Options {
}
}
func (o *Options) LogFile() string {
func (o *options) LogFile() string {
return o.logFile
}
// LogDateTime returns true if the date/time should be displayed in log messages.
func (o *Options) LogDateTime() bool {
func (o *options) LogDateTime() bool {
return o.logDateTime
}
// LogFormat returns the log format.
func (o *Options) LogFormat() string {
func (o *options) LogFormat() string {
return o.logFormat
}
// LogLevel returns the log level.
func (o *Options) LogLevel() string {
func (o *options) LogLevel() string {
return o.logLevel
}
// SetLogLevel sets the log level.
func (o *Options) SetLogLevel(level string) {
func (o *options) SetLogLevel(level string) {
o.logLevel = level
}
// HasMaintenanceMode returns true if maintenance mode is enabled.
func (o *Options) HasMaintenanceMode() bool {
func (o *options) HasMaintenanceMode() bool {
return o.maintenanceMode
}
// MaintenanceMessage returns maintenance message.
func (o *Options) MaintenanceMessage() string {
func (o *options) MaintenanceMessage() string {
return o.maintenanceMessage
}
// BaseURL returns the application base URL with path.
func (o *Options) BaseURL() string {
func (o *options) BaseURL() string {
return o.baseURL
}
// RootURL returns the base URL without path.
func (o *Options) RootURL() string {
func (o *options) RootURL() string {
return o.rootURL
}
// BasePath returns the application base path according to the base URL.
func (o *Options) BasePath() string {
func (o *options) BasePath() string {
return o.basePath
}
// IsDefaultDatabaseURL returns true if the default database URL is used.
func (o *Options) IsDefaultDatabaseURL() bool {
func (o *options) IsDefaultDatabaseURL() bool {
return o.databaseURL == defaultDatabaseURL
}
// DatabaseURL returns the database URL.
func (o *Options) DatabaseURL() string {
func (o *options) DatabaseURL() string {
return o.databaseURL
}
// DatabaseMaxConns returns the maximum number of database connections.
func (o *Options) DatabaseMaxConns() int {
func (o *options) DatabaseMaxConns() int {
return o.databaseMaxConns
}
// DatabaseMinConns returns the minimum number of database connections.
func (o *Options) DatabaseMinConns() int {
func (o *options) DatabaseMinConns() int {
return o.databaseMinConns
}
// DatabaseConnectionLifetime returns the maximum amount of time a connection may be reused.
func (o *Options) DatabaseConnectionLifetime() time.Duration {
func (o *options) DatabaseConnectionLifetime() time.Duration {
return time.Duration(o.databaseConnectionLifetime) * time.Minute
}
// ListenAddr returns the listen address for the HTTP server.
func (o *Options) ListenAddr() []string {
func (o *options) ListenAddr() []string {
return o.listenAddr
}
// CertFile returns the SSL certificate filename if any.
func (o *Options) CertFile() string {
func (o *options) CertFile() string {
return o.certFile
}
// CertKeyFile returns the private key filename for custom SSL certificate.
func (o *Options) CertKeyFile() string {
func (o *options) CertKeyFile() string {
return o.certKeyFile
}
// CertDomain returns the domain to use for Let's Encrypt certificate.
func (o *Options) CertDomain() string {
func (o *options) CertDomain() string {
return o.certDomain
}
// CleanupFrequencyHours returns the interval in hours for cleanup jobs.
func (o *Options) CleanupFrequencyHours() int {
func (o *options) CleanupFrequencyHours() int {
return o.cleanupFrequencyHours
}
// CleanupArchiveReadDays returns the number of days after which marking read items as removed.
func (o *Options) CleanupArchiveReadDays() int {
func (o *options) CleanupArchiveReadDays() int {
return o.cleanupArchiveReadDays
}
// CleanupArchiveUnreadDays returns the number of days after which marking unread items as removed.
func (o *Options) CleanupArchiveUnreadDays() int {
func (o *options) CleanupArchiveUnreadDays() int {
return o.cleanupArchiveUnreadDays
}
// CleanupArchiveBatchSize returns the number of entries to archive for each interval.
func (o *Options) CleanupArchiveBatchSize() int {
func (o *options) CleanupArchiveBatchSize() int {
return o.cleanupArchiveBatchSize
}
// CleanupRemoveSessionsDays returns the number of days after which to remove sessions.
func (o *Options) CleanupRemoveSessionsDays() int {
func (o *options) CleanupRemoveSessionsDays() int {
return o.cleanupRemoveSessionsDays
}
// WorkerPoolSize returns the number of background worker.
func (o *Options) WorkerPoolSize() int {
func (o *options) WorkerPoolSize() int {
return o.workerPoolSize
}
// PollingFrequency returns the interval to refresh feeds in the background.
func (o *Options) PollingFrequency() int {
func (o *options) PollingFrequency() int {
return o.pollingFrequency
}
// ForceRefreshInterval returns the force refresh interval
func (o *Options) ForceRefreshInterval() int {
func (o *options) ForceRefreshInterval() int {
return o.forceRefreshInterval
}
// BatchSize returns the number of feeds to send for background processing.
func (o *Options) BatchSize() int {
func (o *options) BatchSize() int {
return o.batchSize
}
// PollingScheduler returns the scheduler used for polling feeds.
func (o *Options) PollingScheduler() string {
func (o *options) PollingScheduler() string {
return o.pollingScheduler
}
// SchedulerEntryFrequencyMaxInterval returns the maximum interval in minutes for the entry frequency scheduler.
func (o *Options) SchedulerEntryFrequencyMaxInterval() int {
func (o *options) SchedulerEntryFrequencyMaxInterval() int {
return o.schedulerEntryFrequencyMaxInterval
}
// SchedulerEntryFrequencyMinInterval returns the minimum interval in minutes for the entry frequency scheduler.
func (o *Options) SchedulerEntryFrequencyMinInterval() int {
func (o *options) SchedulerEntryFrequencyMinInterval() int {
return o.schedulerEntryFrequencyMinInterval
}
// SchedulerEntryFrequencyFactor returns the factor for the entry frequency scheduler.
func (o *Options) SchedulerEntryFrequencyFactor() int {
func (o *options) SchedulerEntryFrequencyFactor() int {
return o.schedulerEntryFrequencyFactor
}
func (o *Options) SchedulerRoundRobinMinInterval() int {
func (o *options) SchedulerRoundRobinMinInterval() int {
return o.schedulerRoundRobinMinInterval
}
func (o *Options) SchedulerRoundRobinMaxInterval() int {
func (o *options) SchedulerRoundRobinMaxInterval() int {
return o.schedulerRoundRobinMaxInterval
}
// PollingParsingErrorLimit returns the limit of errors when to stop polling.
func (o *Options) PollingParsingErrorLimit() int {
func (o *options) PollingParsingErrorLimit() int {
return o.pollingParsingErrorLimit
}
// IsOAuth2UserCreationAllowed returns true if user creation is allowed for OAuth2 users.
func (o *Options) IsOAuth2UserCreationAllowed() bool {
func (o *options) IsOAuth2UserCreationAllowed() bool {
return o.oauth2UserCreationAllowed
}
// OAuth2ClientID returns the OAuth2 Client ID.
func (o *Options) OAuth2ClientID() string {
func (o *options) OAuth2ClientID() string {
return o.oauth2ClientID
}
// OAuth2ClientSecret returns the OAuth2 client secret.
func (o *Options) OAuth2ClientSecret() string {
func (o *options) OAuth2ClientSecret() string {
return o.oauth2ClientSecret
}
// OAuth2RedirectURL returns the OAuth2 redirect URL.
func (o *Options) OAuth2RedirectURL() string {
func (o *options) OAuth2RedirectURL() string {
return o.oauth2RedirectURL
}
// OIDCDiscoveryEndpoint returns the OAuth2 OIDC discovery endpoint.
func (o *Options) OIDCDiscoveryEndpoint() string {
func (o *options) OIDCDiscoveryEndpoint() string {
return o.oidcDiscoveryEndpoint
}
// OIDCProviderName returns the OAuth2 OIDC provider's display name
func (o *Options) OIDCProviderName() string {
func (o *options) OIDCProviderName() string {
return o.oidcProviderName
}
// OAuth2Provider returns the name of the OAuth2 provider configured.
func (o *Options) OAuth2Provider() string {
func (o *options) OAuth2Provider() string {
return o.oauth2Provider
}
// DisableLocalAUth returns true if the local user database should not be used to authenticate users
func (o *Options) DisableLocalAuth() bool {
func (o *options) DisableLocalAuth() bool {
return o.disableLocalAuth
}
// HasHSTS returns true if HTTP Strict Transport Security is enabled.
func (o *Options) HasHSTS() bool {
func (o *options) HasHSTS() bool {
return o.hsts
}
// RunMigrations returns true if the environment variable RUN_MIGRATIONS is not empty.
func (o *Options) RunMigrations() bool {
func (o *options) RunMigrations() bool {
return o.runMigrations
}
// CreateAdmin returns true if the environment variable CREATE_ADMIN is not empty.
func (o *Options) CreateAdmin() bool {
func (o *options) CreateAdmin() bool {
return o.createAdmin
}
// AdminUsername returns the admin username if defined.
func (o *Options) AdminUsername() string {
func (o *options) AdminUsername() string {
return o.adminUsername
}
// AdminPassword returns the admin password if defined.
func (o *Options) AdminPassword() string {
func (o *options) AdminPassword() string {
return o.adminPassword
}
// FetchYouTubeWatchTime returns true if the YouTube video duration
// should be fetched and used as a reading time.
func (o *Options) FetchYouTubeWatchTime() bool {
func (o *options) FetchYouTubeWatchTime() bool {
return o.fetchYouTubeWatchTime
}
// YouTubeApiKey returns the YouTube API key if defined.
func (o *Options) YouTubeApiKey() string {
func (o *options) YouTubeApiKey() string {
return o.youTubeApiKey
}
// YouTubeEmbedUrlOverride returns the YouTube embed URL override if defined.
func (o *Options) YouTubeEmbedUrlOverride() string {
func (o *options) YouTubeEmbedUrlOverride() string {
return o.youTubeEmbedUrlOverride
}
// YouTubeEmbedDomain returns the domain used for YouTube embeds.
func (o *Options) YouTubeEmbedDomain() string {
func (o *options) YouTubeEmbedDomain() string {
if o.youTubeEmbedDomain != "" {
return o.youTubeEmbedDomain
}
@@ -527,154 +526,154 @@ func (o *Options) YouTubeEmbedDomain() string {
// FetchNebulaWatchTime returns true if the Nebula video duration
// should be fetched and used as a reading time.
func (o *Options) FetchNebulaWatchTime() bool {
func (o *options) FetchNebulaWatchTime() bool {
return o.fetchNebulaWatchTime
}
// FetchOdyseeWatchTime returns true if the Odysee video duration
// should be fetched and used as a reading time.
func (o *Options) FetchOdyseeWatchTime() bool {
func (o *options) FetchOdyseeWatchTime() bool {
return o.fetchOdyseeWatchTime
}
// FetchBilibiliWatchTime returns true if the Bilibili video duration
// should be fetched and used as a reading time.
func (o *Options) FetchBilibiliWatchTime() bool {
func (o *options) FetchBilibiliWatchTime() bool {
return o.fetchBilibiliWatchTime
}
// MediaProxyMode returns "none" to never proxy, "http-only" to proxy non-HTTPS, "all" to always proxy.
func (o *Options) MediaProxyMode() string {
func (o *options) MediaProxyMode() string {
return o.mediaProxyMode
}
// MediaProxyResourceTypes returns a slice of resource types to proxy.
func (o *Options) MediaProxyResourceTypes() []string {
func (o *options) MediaProxyResourceTypes() []string {
return o.mediaProxyResourceTypes
}
// MediaCustomProxyURL returns the custom proxy URL for medias.
func (o *Options) MediaCustomProxyURL() string {
func (o *options) MediaCustomProxyURL() string {
return o.mediaProxyCustomURL
}
// MediaProxyHTTPClientTimeout returns the time limit in seconds before the proxy HTTP client cancel the request.
func (o *Options) MediaProxyHTTPClientTimeout() int {
func (o *options) MediaProxyHTTPClientTimeout() int {
return o.mediaProxyHTTPClientTimeout
}
// MediaProxyPrivateKey returns the private key used by the media proxy.
func (o *Options) MediaProxyPrivateKey() []byte {
func (o *options) MediaProxyPrivateKey() []byte {
return o.mediaProxyPrivateKey
}
// HasHTTPService returns true if the HTTP service is enabled.
func (o *Options) HasHTTPService() bool {
func (o *options) HasHTTPService() bool {
return o.httpService
}
// HasSchedulerService returns true if the scheduler service is enabled.
func (o *Options) HasSchedulerService() bool {
func (o *options) HasSchedulerService() bool {
return o.schedulerService
}
// HTTPClientTimeout returns the time limit in seconds before the HTTP client cancel the request.
func (o *Options) HTTPClientTimeout() int {
func (o *options) HTTPClientTimeout() int {
return o.httpClientTimeout
}
// HTTPClientMaxBodySize returns the number of bytes allowed for the HTTP client to transfer.
func (o *Options) HTTPClientMaxBodySize() int64 {
func (o *options) HTTPClientMaxBodySize() int64 {
return o.httpClientMaxBodySize
}
// HTTPClientProxyURL returns the client HTTP proxy URL if configured.
func (o *Options) HTTPClientProxyURL() *url.URL {
func (o *options) HTTPClientProxyURL() *url.URL {
return o.httpClientProxyURL
}
// HasHTTPClientProxyURLConfigured returns true if the client HTTP proxy URL if configured.
func (o *Options) HasHTTPClientProxyURLConfigured() bool {
func (o *options) HasHTTPClientProxyURLConfigured() bool {
return o.httpClientProxyURL != nil
}
// HTTPClientProxies returns the list of proxies.
func (o *Options) HTTPClientProxies() []string {
func (o *options) HTTPClientProxies() []string {
return o.httpClientProxies
}
// HTTPClientProxiesString returns true if the list of rotating proxies are configured.
func (o *Options) HasHTTPClientProxiesConfigured() bool {
func (o *options) HasHTTPClientProxiesConfigured() bool {
return len(o.httpClientProxies) > 0
}
// HTTPServerTimeout returns the time limit in seconds before the HTTP server cancel the request.
func (o *Options) HTTPServerTimeout() int {
func (o *options) HTTPServerTimeout() int {
return o.httpServerTimeout
}
// AuthProxyHeader returns an HTTP header name that contains username for
// authentication using auth proxy.
func (o *Options) AuthProxyHeader() string {
func (o *options) AuthProxyHeader() string {
return o.authProxyHeader
}
// IsAuthProxyUserCreationAllowed returns true if user creation is allowed for
// users authenticated using auth proxy.
func (o *Options) IsAuthProxyUserCreationAllowed() bool {
func (o *options) IsAuthProxyUserCreationAllowed() bool {
return o.authProxyUserCreation
}
// HasMetricsCollector returns true if metrics collection is enabled.
func (o *Options) HasMetricsCollector() bool {
func (o *options) HasMetricsCollector() bool {
return o.metricsCollector
}
// MetricsRefreshInterval returns the refresh interval in seconds.
func (o *Options) MetricsRefreshInterval() int {
func (o *options) MetricsRefreshInterval() int {
return o.metricsRefreshInterval
}
// MetricsAllowedNetworks returns the list of networks allowed to connect to the metrics endpoint.
func (o *Options) MetricsAllowedNetworks() []string {
func (o *options) MetricsAllowedNetworks() []string {
return o.metricsAllowedNetworks
}
func (o *Options) MetricsUsername() string {
func (o *options) MetricsUsername() string {
return o.metricsUsername
}
func (o *Options) MetricsPassword() string {
func (o *options) MetricsPassword() string {
return o.metricsPassword
}
// HTTPClientUserAgent returns the global User-Agent header for miniflux.
func (o *Options) HTTPClientUserAgent() string {
func (o *options) HTTPClientUserAgent() string {
return o.httpClientUserAgent
}
// HasWatchdog returns true if the systemd watchdog is enabled.
func (o *Options) HasWatchdog() bool {
func (o *options) HasWatchdog() bool {
return o.watchdog
}
// InvidiousInstance returns the invidious instance used by miniflux
func (o *Options) InvidiousInstance() string {
func (o *options) InvidiousInstance() string {
return o.invidiousInstance
}
// WebAuthn returns true if WebAuthn logins are supported
func (o *Options) WebAuthn() bool {
func (o *options) WebAuthn() bool {
return o.webAuthn
}
// FilterEntryMaxAgeDays returns the number of days after which entries should be retained.
func (o *Options) FilterEntryMaxAgeDays() int {
func (o *options) FilterEntryMaxAgeDays() int {
return o.filterEntryMaxAgeDays
}
// SortedOptions returns options as a list of key value pairs, sorted by keys.
func (o *Options) SortedOptions(redactSecret bool) []*Option {
func (o *options) SortedOptions(redactSecret bool) []*option {
var clientProxyURLRedacted string
if o.httpClientProxyURL != nil {
if redactSecret {
@@ -784,20 +783,15 @@ func (o *Options) SortedOptions(redactSecret bool) []*Option {
"WEBAUTHN": o.webAuthn,
}
keys := make([]string, 0, len(keyValues))
for key := range keyValues {
keys = append(keys, key)
}
sort.Strings(keys)
var sortedOptions []*Option
for _, key := range keys {
sortedOptions = append(sortedOptions, &Option{Key: key, Value: keyValues[key]})
sortedKeys := slices.Sorted(maps.Keys(keyValues))
var sortedOptions = make([]*option, 0, len(sortedKeys))
for _, key := range sortedKeys {
sortedOptions = append(sortedOptions, &option{Key: key, Value: keyValues[key]})
}
return sortedOptions
}
func (o *Options) String() string {
func (o *options) String() string {
var builder strings.Builder
for _, option := range o.SortedOptions(false) {
+23 -28
View File
@@ -16,20 +16,20 @@ import (
"strings"
)
// Parser handles configuration parsing.
type Parser struct {
opts *Options
// parser handles configuration parsing.
type parser struct {
opts *options
}
// NewParser returns a new Parser.
func NewParser() *Parser {
return &Parser{
func NewParser() *parser {
return &parser{
opts: NewOptions(),
}
}
// ParseEnvironmentVariables loads configuration values from environment variables.
func (p *Parser) ParseEnvironmentVariables() (*Options, error) {
func (p *parser) ParseEnvironmentVariables() (*options, error) {
err := p.parseLines(os.Environ())
if err != nil {
return nil, err
@@ -38,7 +38,7 @@ func (p *Parser) ParseEnvironmentVariables() (*Options, error) {
}
// ParseFile loads configuration values from a local file.
func (p *Parser) ParseFile(filename string) (*Options, error) {
func (p *parser) ParseFile(filename string) (*options, error) {
fp, err := os.Open(filename)
if err != nil {
return nil, err
@@ -52,7 +52,7 @@ func (p *Parser) ParseFile(filename string) (*Options, error) {
return p.opts, nil
}
func (p *Parser) parseFileContent(r io.Reader) (lines []string) {
func (p *parser) parseFileContent(r io.Reader) (lines []string) {
scanner := bufio.NewScanner(r)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
@@ -63,13 +63,15 @@ func (p *Parser) parseFileContent(r io.Reader) (lines []string) {
return lines
}
func (p *Parser) parseLines(lines []string) (err error) {
func (p *parser) parseLines(lines []string) (err error) {
var port string
for _, line := range lines {
fields := strings.SplitN(line, "=", 2)
key := strings.TrimSpace(fields[0])
value := strings.TrimSpace(fields[1])
for lineNum, line := range lines {
key, value, ok := strings.Cut(line, "=")
if !ok {
return fmt.Errorf("config: unable to parse configuration, invalid format on line %d", lineNum)
}
key, value = strings.TrimSpace(key), strings.TrimSpace(value)
switch key {
case "LOG_FILE":
@@ -275,9 +277,7 @@ func parseBaseURL(value string) (string, string, string, error) {
return defaultBaseURL, defaultRootURL, "", nil
}
if value[len(value)-1:] == "/" {
value = value[:len(value)-1]
}
value = strings.TrimSuffix(value, "/")
parsedURL, err := url.Parse(value)
if err != nil {
@@ -333,19 +333,14 @@ func parseStringList(value string, fallback []string) []string {
}
var strList []string
strMap := make(map[string]bool)
present := make(map[string]bool)
items := strings.Split(value, ",")
for _, item := range items {
itemValue := strings.TrimSpace(item)
if itemValue == "" {
continue
}
if _, found := strMap[itemValue]; !found {
strMap[itemValue] = true
strList = append(strList, itemValue)
for item := range strings.SplitSeq(value, ",") {
if itemValue := strings.TrimSpace(item); itemValue != "" {
if !present[itemValue] {
present[itemValue] = true
strList = append(strList, itemValue)
}
}
}
+1 -3
View File
@@ -14,11 +14,9 @@ func Migrate(db *sql.DB) error {
var currentVersion int
db.QueryRow(`SELECT version FROM schema_version`).Scan(&currentVersion)
driver := getDriverStr()
slog.Info("Running database migrations",
slog.Int("current_version", currentVersion),
slog.Int("latest_version", schemaVersion),
slog.String("driver", driver),
)
for version := currentVersion; version < schemaVersion; version++ {
@@ -29,7 +27,7 @@ func Migrate(db *sql.DB) error {
return fmt.Errorf("[Migration v%d] %v", newVersion, err)
}
if err := migrations[version](tx, driver); err != nil {
if err := migrations[version](tx); err != nil {
tx.Rollback()
return fmt.Errorf("[Migration v%d] %v", newVersion, err)
}
+143 -150
View File
@@ -12,8 +12,8 @@ import (
var schemaVersion = len(migrations)
// Order is important. Add new migrations at the end of the list.
var migrations = []func(tx *sql.Tx, driver string) error{
func(tx *sql.Tx, _ string) (err error) {
var migrations = []func(tx *sql.Tx) error{
func(tx *sql.Tx) (err error) {
sql := `
CREATE TABLE schema_version (
version text not null
@@ -122,19 +122,16 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, driver string) (err error) {
if driver == "postgresql" {
sql := `
func(tx *sql.Tx) (err error) {
sql := `
CREATE EXTENSION IF NOT EXISTS hstore;
ALTER TABLE users ADD COLUMN extra hstore;
CREATE INDEX users_extra_idx ON users using gin(extra);
`
_, err = tx.Exec(sql)
return err
}
return nil
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
CREATE TABLE tokens (
id text not null,
@@ -146,7 +143,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
CREATE TYPE entry_sorting_direction AS enum('asc', 'desc');
ALTER TABLE users ADD COLUMN entry_direction entry_sorting_direction default 'asc';
@@ -154,7 +151,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
CREATE TABLE integrations (
user_id int not null,
@@ -175,27 +172,27 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN scraper_rules text default ''`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN rewrite_rules text default ''`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN crawler boolean default 'f'`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE sessions rename to user_sessions`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
DROP TABLE tokens;
@@ -209,7 +206,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN wallabag_enabled bool default 'f',
@@ -222,12 +219,12 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE entries ADD COLUMN starred bool default 'f'`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
CREATE INDEX entries_user_status_idx ON entries(user_id, status);
CREATE INDEX feeds_user_category_idx ON feeds(user_id, category_id);
@@ -235,7 +232,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN nunux_keeper_enabled bool default 'f',
@@ -245,17 +242,17 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE enclosures ALTER COLUMN size SET DATA TYPE bigint`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE entries ADD COLUMN comments_url text default ''`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN pocket_enabled bool default 'f',
@@ -265,14 +262,14 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE user_sessions ALTER COLUMN ip SET DATA TYPE inet using ip::inet;
`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE feeds
ADD COLUMN username text default '',
@@ -281,7 +278,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE entries ADD COLUMN document_vectors tsvector;
UPDATE entries SET document_vectors = to_tsvector(substring(title || ' ' || coalesce(content, '') for 1000000));
@@ -290,12 +287,12 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN user_agent text default ''`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
UPDATE
entries
@@ -305,17 +302,17 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN keyboard_shortcuts boolean default 't'`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN disabled boolean default 'f';`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE users ALTER COLUMN theme SET DEFAULT 'light_serif';
UPDATE users SET theme='light_serif' WHERE theme='default';
@@ -325,7 +322,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE entries ADD COLUMN changed_at timestamp with time zone;
UPDATE entries SET changed_at = published_at;
@@ -334,7 +331,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
CREATE TABLE api_keys (
id serial not null,
@@ -350,7 +347,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE entries ADD COLUMN share_code text not null default '';
CREATE UNIQUE INDEX entries_share_code_idx ON entries USING btree(share_code) WHERE share_code <> '';
@@ -358,12 +355,12 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `CREATE INDEX enclosures_user_entry_url_idx ON enclosures(user_id, entry_id, md5(url))`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE feeds ADD COLUMN next_check_at timestamp with time zone default now();
CREATE INDEX entries_user_feed_idx ON entries (user_id, feed_id);
@@ -371,52 +368,52 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN ignore_http_cache bool default false`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN entries_per_page int default 100`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN show_reading_time boolean default 't'`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `CREATE INDEX entries_id_user_status_idx ON entries USING btree (id, user_id, status)`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN fetch_via_proxy bool default false`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `CREATE INDEX entries_feed_id_status_hash_idx ON entries USING btree (feed_id, status, hash)`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `CREATE INDEX entries_user_id_status_starred_idx ON entries (user_id, status, starred)`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN entry_swipe boolean default 't'`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE integrations DROP COLUMN fever_password`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE feeds
ADD COLUMN blocklist_rules text not null default '',
@@ -425,12 +422,12 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE entries ADD COLUMN reading_time int not null default 0`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE entries ADD COLUMN created_at timestamp with time zone not null default now();
UPDATE entries SET created_at = published_at;
@@ -438,7 +435,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, driver string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE users
ADD column stylesheet text not null default '',
@@ -449,8 +446,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
return err
}
if driver == "postgresql" {
_, err = tx.Exec(`
_, err = tx.Exec(`
DECLARE my_cursor CURSOR FOR
SELECT
id,
@@ -460,28 +456,28 @@ var migrations = []func(tx *sql.Tx, driver string) error{
FROM users
FOR UPDATE
`)
if err != nil {
if err != nil {
return err
}
defer tx.Exec("CLOSE my_cursor")
for {
var (
userID int64
customStylesheet string
googleID string
oidcID string
)
if err := tx.QueryRow(`FETCH NEXT FROM my_cursor`).Scan(&userID, &customStylesheet, &googleID, &oidcID); err != nil {
if err == sql.ErrNoRows {
break
}
return err
}
defer tx.Exec("CLOSE my_cursor")
for {
var (
userID int64
customStylesheet string
googleID string
oidcID string
)
if err := tx.QueryRow(`FETCH NEXT FROM my_cursor`).Scan(&userID, &customStylesheet, &googleID, &oidcID); err != nil {
if err == sql.ErrNoRows {
break
}
return err
}
_, err := tx.Exec(
`UPDATE
_, err := tx.Exec(
`UPDATE
users
SET
stylesheet=$2,
@@ -490,20 +486,17 @@ var migrations = []func(tx *sql.Tx, driver string) error{
WHERE
id=$1
`,
userID, customStylesheet, googleID, oidcID)
if err != nil {
return err
}
userID, customStylesheet, googleID, oidcID)
if err != nil {
return err
}
}
return err
},
func(tx *sql.Tx, driver string) (err error) {
if driver == "postgresql" {
if _, err = tx.Exec(`ALTER TABLE users DROP COLUMN extra;`); err != nil {
return err
}
func(tx *sql.Tx) (err error) {
if _, err = tx.Exec(`ALTER TABLE users DROP COLUMN extra;`); err != nil {
return err
}
_, err = tx.Exec(`
CREATE UNIQUE INDEX users_google_id_idx ON users(google_id) WHERE google_id <> '';
@@ -511,7 +504,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
CREATE INDEX entries_feed_url_idx ON entries(feed_id, url) WHERE length(url) < 2000;
CREATE INDEX entries_user_status_feed_idx ON entries(user_id, status, feed_id);
@@ -519,7 +512,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
CREATE TABLE acme_cache (
key varchar(400) not null primary key,
@@ -529,13 +522,13 @@ var migrations = []func(tx *sql.Tx, driver string) error{
`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE feeds ADD COLUMN allow_self_signed_certificates boolean not null default false
`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
CREATE TYPE webapp_display_mode AS enum('fullscreen', 'standalone', 'minimal-ui', 'browser');
ALTER TABLE users ADD COLUMN display_mode webapp_display_mode default 'standalone';
@@ -543,24 +536,24 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN cookie text default ''`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE categories ADD COLUMN hide_globally boolean not null default false
`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE feeds ADD COLUMN hide_globally boolean not null default false
`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN telegram_bot_enabled bool default 'f',
@@ -570,7 +563,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
CREATE TYPE entry_sorting_order AS enum('published_at', 'created_at');
ALTER TABLE users ADD COLUMN entry_order entry_sorting_order default 'published_at';
@@ -578,7 +571,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN googlereader_enabled bool default 'f',
@@ -588,7 +581,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN espial_enabled bool default 'f',
@@ -599,7 +592,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN linkding_enabled bool default 'f',
@@ -609,13 +602,13 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE feeds ADD COLUMN url_rewrite_rules text not null default ''
`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE users
ADD COLUMN default_reading_speed int default 265,
@@ -623,25 +616,25 @@ var migrations = []func(tx *sql.Tx, driver string) error{
`)
return
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE users ADD COLUMN default_home_page text default 'unread';
`)
return
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE integrations ADD COLUMN wallabag_only_url bool default 'f';
`)
return
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE users ADD COLUMN categories_sorting_order text not null default 'unread_count';
`)
return
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN matrix_bot_enabled bool default 'f',
@@ -653,18 +646,18 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN double_tap boolean default 't'`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE entries ADD COLUMN tags text[] default '{}';
`)
return
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE users RENAME double_tap TO gesture_nav;
ALTER TABLE users
@@ -674,14 +667,14 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations ADD COLUMN linkding_tags text default '';
`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE feeds ADD COLUMN no_media_player boolean default 'f';
ALTER TABLE enclosures ADD COLUMN media_progression int default 0;
@@ -689,14 +682,14 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations ADD COLUMN linkding_mark_as_unread bool default 'f';
`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
// Delete duplicated rows
sql := `
DELETE FROM enclosures a USING enclosures b
@@ -724,12 +717,12 @@ var migrations = []func(tx *sql.Tx, driver string) error{
return nil
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN mark_read_on_view boolean default 't'`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN notion_enabled bool default 'f',
@@ -739,7 +732,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN readwise_enabled bool default 'f',
@@ -748,7 +741,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN apprise_enabled bool default 'f',
@@ -758,7 +751,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN shiori_enabled bool default 'f',
@@ -769,7 +762,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN shaarli_enabled bool default 'f',
@@ -779,13 +772,13 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
ALTER TABLE feeds ADD COLUMN apprise_service_urls text default '';
`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN webhook_enabled bool default 'f',
@@ -795,7 +788,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN telegram_bot_topic_id int,
@@ -805,14 +798,14 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations ADD COLUMN telegram_bot_disable_buttons bool default 'f';
`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
-- Speed up has_enclosure
CREATE INDEX enclosures_entry_id_idx ON enclosures(entry_id);
@@ -828,7 +821,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN rssbridge_enabled bool default 'f',
@@ -837,7 +830,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
CREATE TABLE webauthn_credentials (
handle bytea primary key,
@@ -855,7 +848,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
`)
return
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN omnivore_enabled bool default 'f',
@@ -865,7 +858,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN linkace_enabled bool default 'f',
@@ -878,7 +871,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN linkwarden_enabled bool default 'f',
@@ -888,7 +881,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN readeck_enabled bool default 'f',
@@ -900,29 +893,29 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN disable_http2 bool default 'f'`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN media_playback_rate numeric default 1;`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
// the WHERE part speed-up the request a lot
sql := `UPDATE entries SET tags = array_remove(tags, '') WHERE '' = ANY(tags);`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
// Entry URLs can exceeds btree maximum size
// Checking entry existence is now using entries_feed_id_status_hash_idx index
_, err = tx.Exec(`DROP INDEX entries_feed_url_idx`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN raindrop_enabled bool default 'f',
@@ -933,12 +926,12 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE feeds ADD COLUMN description text default ''`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE users
ADD COLUMN block_filter_entry_rules text not null default '',
@@ -947,7 +940,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN betula_url text default '',
@@ -957,7 +950,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN ntfy_enabled bool default 'f',
@@ -975,22 +968,22 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN mark_read_on_media_player_completion bool default 'f';`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN custom_js text not null default '';`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE users ADD COLUMN external_font_hosts text not null default '';`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN cubox_enabled bool default 'f',
@@ -999,7 +992,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN discord_enabled bool default 'f',
@@ -1008,12 +1001,12 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `ALTER TABLE integrations ADD COLUMN ntfy_internal_links bool default 'f';`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN slack_enabled bool default 'f',
@@ -1022,11 +1015,11 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`ALTER TABLE feeds ADD COLUMN webhook_url text default '';`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN pushover_enabled bool default 'f',
@@ -1042,14 +1035,14 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE feeds ADD COLUMN ntfy_topic text default '';
`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE icons ADD COLUMN external_id text default '';
CREATE UNIQUE INDEX icons_external_id_idx ON icons USING btree(external_id) WHERE external_id <> '';
@@ -1058,7 +1051,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`
DECLARE id_cursor CURSOR FOR
SELECT
@@ -1094,22 +1087,22 @@ var migrations = []func(tx *sql.Tx, driver string) error{
return nil
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`ALTER TABLE feeds ADD COLUMN proxy_url text default ''`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations ADD COLUMN rssbridge_token text default '';
`
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`ALTER TABLE users ADD COLUMN always_open_external_links bool default 'f'`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
ADD COLUMN karakeep_enabled bool default 'f',
@@ -1119,11 +1112,11 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
_, err = tx.Exec(`ALTER TABLE users ADD COLUMN open_external_links_in_new_tab bool default 't'`)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE integrations
DROP COLUMN pocket_enabled,
@@ -1133,7 +1126,7 @@ var migrations = []func(tx *sql.Tx, driver string) error{
_, err = tx.Exec(sql)
return err
},
func(tx *sql.Tx, _ string) (err error) {
func(tx *sql.Tx) (err error) {
sql := `
ALTER TABLE feeds
ADD COLUMN block_filter_entry_rules text not null default '',
-6
View File
@@ -1,5 +1,3 @@
//go:build !sqlite
// SPDX-FileCopyrightText: Copyright The Miniflux Authors. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
@@ -25,7 +23,3 @@ func NewConnectionPool(dsn string, minConnections, maxConnections int, connectio
return db, nil
}
func getDriverStr() string {
return "postgresql"
}
-26
View File
@@ -1,26 +0,0 @@
//go:build sqlite
// SPDX-FileCopyrightText: Copyright The Miniflux Authors. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package database // import "miniflux.app/v2/internal/database"
import (
"database/sql"
"time"
_ "github.com/mattn/go-sqlite3"
)
// NewConnectionPool configures the database connection pool.
func NewConnectionPool(dsn string, _, _ int, _ time.Duration) (*sql.DB, error) {
db, err := sql.Open("sqlite3", dsn)
if err != nil {
return nil, err
}
return db, nil
}
func getDriverStr() string {
return "sqlite3"
}
+46 -47
View File
@@ -71,12 +71,12 @@ func checkAndSimplifyTags(addTags []Stream, removeTags []Stream) (map[StreamType
switch s.Type {
case ReadStream:
if _, ok := tags[KeptUnreadStream]; ok {
return nil, fmt.Errorf("googlereader: %s and %s should not be supplied simultaneously", KeptUnread, Read)
return nil, fmt.Errorf("googlereader: %s and %s should not be supplied simultaneously", keptUnreadStreamSuffix, readStreamSuffix)
}
tags[ReadStream] = true
case KeptUnreadStream:
if _, ok := tags[ReadStream]; ok {
return nil, fmt.Errorf("googlereader: %s and %s should not be supplied simultaneously", KeptUnread, Read)
return nil, fmt.Errorf("googlereader: %s and %s should not be supplied simultaneously", keptUnreadStreamSuffix, readStreamSuffix)
}
tags[ReadStream] = false
case StarredStream:
@@ -91,17 +91,17 @@ func checkAndSimplifyTags(addTags []Stream, removeTags []Stream) (map[StreamType
switch s.Type {
case ReadStream:
if _, ok := tags[ReadStream]; ok {
return nil, fmt.Errorf("googlereader: %s and %s should not be supplied simultaneously", KeptUnread, Read)
return nil, fmt.Errorf("googlereader: %s and %s should not be supplied simultaneously", keptUnreadStreamSuffix, readStreamSuffix)
}
tags[ReadStream] = false
case KeptUnreadStream:
if _, ok := tags[ReadStream]; ok {
return nil, fmt.Errorf("googlereader: %s and %s should not be supplied simultaneously", KeptUnread, Read)
return nil, fmt.Errorf("googlereader: %s and %s should not be supplied simultaneously", keptUnreadStreamSuffix, readStreamSuffix)
}
tags[ReadStream] = true
case StarredStream:
if _, ok := tags[StarredStream]; ok {
return nil, fmt.Errorf("googlereader: %s should not be supplied for add and remove simultaneously", Starred)
return nil, fmt.Errorf("googlereader: %s should not be supplied for add and remove simultaneously", starredStreamSuffix)
}
tags[StarredStream] = false
case BroadcastStream, LikeStream:
@@ -200,7 +200,7 @@ func (h *handler) clientLoginHandler(w http.ResponseWriter, r *http.Request) {
slog.String("username", username),
)
result := login{SID: token, LSID: token, Auth: token}
result := loginResponse{SID: token, LSID: token, Auth: token}
if output == "json" {
json.OK(w, r, result)
return
@@ -269,12 +269,12 @@ func (h *handler) editTagHandler(w http.ResponseWriter, r *http.Request) {
return
}
addTags, err := getStreams(r.PostForm[ParamTagsAdd], userID)
addTags, err := getStreams(r.PostForm[paramTagsAdd], userID)
if err != nil {
json.ServerError(w, r, err)
return
}
removeTags, err := getStreams(r.PostForm[ParamTagsRemove], userID)
removeTags, err := getStreams(r.PostForm[paramTagsRemove], userID)
if err != nil {
json.ServerError(w, r, err)
return
@@ -387,7 +387,7 @@ func (h *handler) editTagHandler(w http.ResponseWriter, r *http.Request) {
}
}
OK(w, r)
sendOkayResponse(w)
}
func (h *handler) quickAddHandler(w http.ResponseWriter, r *http.Request) {
@@ -407,7 +407,7 @@ func (h *handler) quickAddHandler(w http.ResponseWriter, r *http.Request) {
return
}
feedURL := r.Form.Get(ParamQuickAdd)
feedURL := r.Form.Get(paramQuickAdd)
if !validator.IsValidURL(feedURL) {
json.BadRequest(w, r, fmt.Errorf("googlereader: invalid URL: %s", feedURL))
return
@@ -456,7 +456,7 @@ func (h *handler) quickAddHandler(w http.ResponseWriter, r *http.Request) {
json.OK(w, r, quickAddResponse{
NumResults: 1,
Query: newFeed.FeedURL,
StreamID: fmt.Sprintf(FeedPrefix+"%d", newFeed.ID),
StreamID: fmt.Sprintf(feedPrefix+"%d", newFeed.ID),
StreamName: newFeed.Title,
})
}
@@ -609,20 +609,20 @@ func (h *handler) editSubscriptionHandler(w http.ResponseWriter, r *http.Request
return
}
streamIds, err := getStreams(r.Form[ParamStreamID], userID)
streamIds, err := getStreams(r.Form[paramStreamID], userID)
if err != nil || len(streamIds) == 0 {
json.BadRequest(w, r, errors.New("googlereader: no valid stream IDs provided"))
return
}
newLabel, err := getStream(r.Form.Get(ParamTagsAdd), userID)
newLabel, err := getStream(r.Form.Get(paramTagsAdd), userID)
if err != nil {
json.BadRequest(w, r, fmt.Errorf("googlereader: invalid data in %s", ParamTagsAdd))
json.BadRequest(w, r, fmt.Errorf("googlereader: invalid data in %s", paramTagsAdd))
return
}
title := r.Form.Get(ParamTitle)
action := r.Form.Get(ParamSubscribeAction)
title := r.Form.Get(paramTitle)
action := r.Form.Get(paramSubscribeAction)
switch action {
case "subscribe":
@@ -649,7 +649,7 @@ func (h *handler) editSubscriptionHandler(w http.ResponseWriter, r *http.Request
}
}
if r.Form.Has(ParamTagsAdd) {
if r.Form.Has(paramTagsAdd) {
if newLabel.Type != LabelStream {
json.BadRequest(w, r, errors.New("destination must be a label"))
return
@@ -669,7 +669,7 @@ func (h *handler) editSubscriptionHandler(w http.ResponseWriter, r *http.Request
return
}
OK(w, r)
sendOkayResponse(w)
}
func (h *handler) streamItemContentsHandler(w http.ResponseWriter, r *http.Request) {
@@ -701,9 +701,9 @@ func (h *handler) streamItemContentsHandler(w http.ResponseWriter, r *http.Reque
return
}
userReadingList := fmt.Sprintf(UserStreamPrefix, userID) + ReadingList
userRead := fmt.Sprintf(UserStreamPrefix, userID) + Read
userStarred := fmt.Sprintf(UserStreamPrefix, userID) + Starred
userReadingList := fmt.Sprintf(userStreamPrefix, userID) + readingListStreamSuffix
userRead := fmt.Sprintf(userStreamPrefix, userID) + readStreamSuffix
userStarred := fmt.Sprintf(userStreamPrefix, userID) + starredStreamSuffix
itemIDs, err := parseItemIDsFromRequest(r)
if err != nil {
@@ -731,7 +731,7 @@ func (h *handler) streamItemContentsHandler(w http.ResponseWriter, r *http.Reque
return
}
result := streamContentItems{
result := streamContentItemsResponse{
Direction: "ltr",
ID: "user/-/state/com.google/reading-list",
Title: "Reading List",
@@ -752,7 +752,7 @@ func (h *handler) streamItemContentsHandler(w http.ResponseWriter, r *http.Reque
categories := make([]string, 0)
categories = append(categories, userReadingList)
if entry.Feed.Category.Title != "" {
categories = append(categories, fmt.Sprintf(UserLabelPrefix, userID)+entry.Feed.Category.Title)
categories = append(categories, fmt.Sprintf(userLabelPrefix, userID)+entry.Feed.Category.Title)
}
if entry.Status == model.EntryStatusRead {
categories = append(categories, userRead)
@@ -763,8 +763,7 @@ func (h *handler) streamItemContentsHandler(w http.ResponseWriter, r *http.Reque
}
entry.Content = mediaproxy.RewriteDocumentWithAbsoluteProxyURL(h.router, entry.Content)
entry.Enclosures.ProxifyEnclosureURL(h.router)
entry.Enclosures.ProxifyEnclosureURL(h.router, config.Opts.MediaProxyMode(), config.Opts.MediaProxyResourceTypes())
contentItems[i] = contentItem{
ID: convertEntryIDToLongFormItemID(entry.ID),
@@ -823,9 +822,9 @@ func (h *handler) disableTagHandler(w http.ResponseWriter, r *http.Request) {
return
}
streams, err := getStreams(r.Form[ParamStreamID], userID)
streams, err := getStreams(r.Form[paramStreamID], userID)
if err != nil {
json.BadRequest(w, r, fmt.Errorf("googlereader: invalid data in %s", ParamStreamID))
json.BadRequest(w, r, fmt.Errorf("googlereader: invalid data in %s", paramStreamID))
return
}
@@ -844,7 +843,7 @@ func (h *handler) disableTagHandler(w http.ResponseWriter, r *http.Request) {
return
}
OK(w, r)
sendOkayResponse(w)
}
func (h *handler) renameTagHandler(w http.ResponseWriter, r *http.Request) {
@@ -863,15 +862,15 @@ func (h *handler) renameTagHandler(w http.ResponseWriter, r *http.Request) {
return
}
source, err := getStream(r.Form.Get(ParamStreamID), userID)
source, err := getStream(r.Form.Get(paramStreamID), userID)
if err != nil {
json.BadRequest(w, r, fmt.Errorf("googlereader: invalid data in %s", ParamStreamID))
json.BadRequest(w, r, fmt.Errorf("googlereader: invalid data in %s", paramStreamID))
return
}
destination, err := getStream(r.Form.Get(ParamDestination), userID)
destination, err := getStream(r.Form.Get(paramDestination), userID)
if err != nil {
json.BadRequest(w, r, fmt.Errorf("googlereader: invalid data in %s", ParamDestination))
json.BadRequest(w, r, fmt.Errorf("googlereader: invalid data in %s", paramDestination))
return
}
@@ -911,7 +910,7 @@ func (h *handler) renameTagHandler(w http.ResponseWriter, r *http.Request) {
return
}
OK(w, r)
sendOkayResponse(w)
}
func (h *handler) tagListHandler(w http.ResponseWriter, r *http.Request) {
@@ -935,13 +934,13 @@ func (h *handler) tagListHandler(w http.ResponseWriter, r *http.Request) {
json.ServerError(w, r, err)
return
}
result.Tags = make([]subscriptionCategory, 0)
result.Tags = append(result.Tags, subscriptionCategory{
ID: fmt.Sprintf(UserStreamPrefix, userID) + Starred,
result.Tags = make([]subscriptionCategoryResponse, 0)
result.Tags = append(result.Tags, subscriptionCategoryResponse{
ID: fmt.Sprintf(userStreamPrefix, userID) + starredStreamSuffix,
})
for _, category := range categories {
result.Tags = append(result.Tags, subscriptionCategory{
ID: fmt.Sprintf(UserLabelPrefix, userID) + category.Title,
result.Tags = append(result.Tags, subscriptionCategoryResponse{
ID: fmt.Sprintf(userLabelPrefix, userID) + category.Title,
Label: category.Title,
Type: "folder",
})
@@ -971,13 +970,13 @@ func (h *handler) subscriptionListHandler(w http.ResponseWriter, r *http.Request
return
}
result.Subscriptions = make([]subscription, 0)
result.Subscriptions = make([]subscriptionResponse, 0)
for _, feed := range feeds {
result.Subscriptions = append(result.Subscriptions, subscription{
ID: fmt.Sprintf(FeedPrefix+"%d", feed.ID),
result.Subscriptions = append(result.Subscriptions, subscriptionResponse{
ID: fmt.Sprintf(feedPrefix+"%d", feed.ID),
Title: feed.Title,
URL: feed.FeedURL,
Categories: []subscriptionCategory{{fmt.Sprintf(UserLabelPrefix, userID) + feed.Category.Title, feed.Category.Title, "folder"}},
Categories: []subscriptionCategoryResponse{{fmt.Sprintf(userLabelPrefix, userID) + feed.Category.Title, feed.Category.Title, "folder"}},
HTMLURL: feed.SiteURL,
IconURL: h.feedIconURL(feed),
})
@@ -1016,7 +1015,7 @@ func (h *handler) userInfoHandler(w http.ResponseWriter, r *http.Request) {
json.ServerError(w, r, err)
return
}
userInfo := userInfo{UserID: fmt.Sprint(user.ID), UserName: user.Username, UserProfileID: fmt.Sprint(user.ID), UserEmail: user.Username}
userInfo := userInfoResponse{UserID: fmt.Sprint(user.ID), UserName: user.Username, UserProfileID: fmt.Sprint(user.ID), UserEmail: user.Username}
json.OK(w, r, userInfo)
}
@@ -1092,7 +1091,7 @@ func (h *handler) handleReadingListStreamHandler(w http.ResponseWriter, r *http.
slog.String("handler", "handleReadingListStreamHandler"),
slog.String("client_ip", clientIP),
slog.String("user_agent", r.UserAgent()),
slog.Any("filter_type", s.Type),
slog.Int("filter_type", int(s.Type)),
)
}
}
@@ -1279,14 +1278,14 @@ func (h *handler) markAllAsReadHandler(w http.ResponseWriter, r *http.Request) {
return
}
stream, err := getStream(r.Form.Get(ParamStreamID), userID)
stream, err := getStream(r.Form.Get(paramStreamID), userID)
if err != nil {
json.BadRequest(w, r, err)
return
}
var before time.Time
if timestampParamValue := r.Form.Get(ParamTimestamp); timestampParamValue != "" {
if timestampParamValue := r.Form.Get(paramTimestamp); timestampParamValue != "" {
timestampParsedValue, err := strconv.ParseInt(timestampParamValue, 10, 64)
if err != nil {
json.BadRequest(w, r, err)
@@ -1340,5 +1339,5 @@ func (h *handler) markAllAsReadHandler(w http.ResponseWriter, r *http.Request) {
}
}
OK(w, r)
sendOkayResponse(w)
}
+1 -1
View File
@@ -56,7 +56,7 @@ func parseItemID(itemIDValue string) (int64, error) {
}
func parseItemIDsFromRequest(r *http.Request) ([]int64, error) {
items := r.Form[ParamItemIDs]
items := r.Form[paramItemIDs]
if len(items) == 0 {
return nil, fmt.Errorf("googlereader: no items requested")
}
+12 -12
View File
@@ -51,7 +51,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("user_agent", r.UserAgent()),
slog.Any("error", err),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
@@ -62,7 +62,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("client_ip", clientIP),
slog.String("user_agent", r.UserAgent()),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
} else {
@@ -74,7 +74,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("client_ip", clientIP),
slog.String("user_agent", r.UserAgent()),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
fields := strings.Fields(authorization)
@@ -84,7 +84,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("client_ip", clientIP),
slog.String("user_agent", r.UserAgent()),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
if fields[0] != "GoogleLogin" {
@@ -93,7 +93,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("client_ip", clientIP),
slog.String("user_agent", r.UserAgent()),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
auths := strings.Split(fields[1], "=")
@@ -103,7 +103,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("client_ip", clientIP),
slog.String("user_agent", r.UserAgent()),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
if auths[0] != "auth" {
@@ -112,7 +112,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("client_ip", clientIP),
slog.String("user_agent", r.UserAgent()),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
token = auths[1]
@@ -126,7 +126,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("user_agent", r.UserAgent()),
slog.String("token", token),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
var integration *model.Integration
@@ -139,7 +139,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("user_agent", r.UserAgent()),
slog.Any("error", err),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
expectedToken := getAuthToken(integration.GoogleReaderUsername, integration.GoogleReaderPassword)
@@ -149,7 +149,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("client_ip", clientIP),
slog.String("user_agent", r.UserAgent()),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
if user, err = m.store.UserByID(integration.UserID); err != nil {
@@ -159,7 +159,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("user_agent", r.UserAgent()),
slog.Any("error", err),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
@@ -169,7 +169,7 @@ func (m *middleware) apiKeyAuth(next http.Handler) http.Handler {
slog.String("client_ip", clientIP),
slog.String("user_agent", r.UserAgent()),
)
Unauthorized(w, r)
sendUnauthorizedResponse(w)
return
}
+32 -32
View File
@@ -4,36 +4,36 @@
package googlereader // import "miniflux.app/v2/internal/googlereader"
const (
// ParamItemIDs - name of the parameter with the item ids
ParamItemIDs = "i"
// ParamStreamID - name of the parameter containing the stream to be included
ParamStreamID = "s"
// ParamStreamExcludes - name of the parameter containing streams to be excluded
ParamStreamExcludes = "xt"
// ParamStreamFilters - name of the parameter containing streams to be included
ParamStreamFilters = "it"
// ParamStreamMaxItems - name of the parameter containing number of items per page/max items returned
ParamStreamMaxItems = "n"
// ParamStreamOrder - name of the parameter containing the sort criteria
ParamStreamOrder = "r"
// ParamStreamStartTime - name of the parameter containing epoch timestamp, filtering items older than
ParamStreamStartTime = "ot"
// ParamStreamStopTime - name of the parameter containing epoch timestamp, filtering items newer than
ParamStreamStopTime = "nt"
// ParamTagsRemove - name of the parameter containing tags (streams) to be removed
ParamTagsRemove = "r"
// ParamTagsAdd - name of the parameter containing tags (streams) to be added
ParamTagsAdd = "a"
// ParamSubscribeAction - name of the parameter indicating the action to take for subscription/edit
ParamSubscribeAction = "ac"
// ParamTitle - name of the parameter for the title of the subscription
ParamTitle = "t"
// ParamQuickAdd - name of the parameter for a URL being quick subscribed to
ParamQuickAdd = "quickadd"
// ParamDestination - name of the parameter for the new name of a tag
ParamDestination = "dest"
// ParamContinuation - name of the parameter for callers to pass to receive the next page of results
ParamContinuation = "c"
// ParamStreamType - name of the parameter for unix timestamp
ParamTimestamp = "ts"
// paramItemIDs - name of the parameter with the item ids
paramItemIDs = "i"
// paramStreamID - name of the parameter containing the stream to be included
paramStreamID = "s"
// paramStreamExcludes - name of the parameter containing streams to be excluded
paramStreamExcludes = "xt"
// paramStreamFilters - name of the parameter containing streams to be included
paramStreamFilters = "it"
// paramStreamMaxItems - name of the parameter containing number of items per page/max items returned
paramStreamMaxItems = "n"
// paramStreamOrder - name of the parameter containing the sort criteria
paramStreamOrder = "r"
// paramStreamStartTime - name of the parameter containing epoch timestamp, filtering items older than
paramStreamStartTime = "ot"
// paramStreamStopTime - name of the parameter containing epoch timestamp, filtering items newer than
paramStreamStopTime = "nt"
// paramTagsRemove - name of the parameter containing tags (streams) to be removed
paramTagsRemove = "r"
// paramTagsAdd - name of the parameter containing tags (streams) to be added
paramTagsAdd = "a"
// paramSubscribeAction - name of the parameter indicating the action to take for subscription/edit
paramSubscribeAction = "ac"
// paramTitle - name of the parameter for the title of the subscription
paramTitle = "t"
// paramQuickAdd - name of the parameter for a URL being quick subscribed to
paramQuickAdd = "quickadd"
// paramDestination - name of the parameter for the new name of a tag
paramDestination = "dest"
// paramContinuation - name of the parameter for callers to pass to receive the next page of results
paramContinuation = "c"
// paramTimestamp - name of the parameter for unix timestamp
paramTimestamp = "ts"
)
+24 -24
View File
@@ -4,28 +4,28 @@
package googlereader // import "miniflux.app/v2/internal/googlereader"
const (
// StreamPrefix is the prefix for astreams (read/starred/reading list and so on)
StreamPrefix = "user/-/state/com.google/"
// UserStreamPrefix is the user specific prefix for streams (read/starred/reading list and so on)
UserStreamPrefix = "user/%d/state/com.google/"
// LabelPrefix is the prefix for a label stream
LabelPrefix = "user/-/label/"
// UserLabelPrefix is the user specific prefix prefix for a label stream
UserLabelPrefix = "user/%d/label/"
// FeedPrefix is the prefix for a feed stream
FeedPrefix = "feed/"
// Read is the suffix for read stream
Read = "read"
// Starred is the suffix for starred stream
Starred = "starred"
// ReadingList is the suffix for reading list stream
ReadingList = "reading-list"
// KeptUnread is the suffix for kept unread stream
KeptUnread = "kept-unread"
// Broadcast is the suffix for broadcast stream
Broadcast = "broadcast"
// BroadcastFriends is the suffix for broadcast friends stream
BroadcastFriends = "broadcast-friends"
// Like is the suffix for like stream
Like = "like"
// streamPrefix is the prefix for streams (read/starred/reading list and so on)
streamPrefix = "user/-/state/com.google/"
// userStreamPrefix is the user specific prefix for streams (read/starred/reading list and so on)
userStreamPrefix = "user/%d/state/com.google/"
// labelPrefix is the prefix for a label stream
labelPrefix = "user/-/label/"
// userLabelPrefix is the user specific prefix prefix for a label stream
userLabelPrefix = "user/%d/label/"
// feedPrefix is the prefix for a feed stream
feedPrefix = "feed/"
// readStreamSuffix is the suffix for read stream
readStreamSuffix = "read"
// starredStreamSuffix is the suffix for starred stream
starredStreamSuffix = "starred"
// readingListStreamSuffix is the suffix for reading list stream
readingListStreamSuffix = "reading-list"
// keptUnreadStreamSuffix is the suffix for kept unread stream
keptUnreadStreamSuffix = "kept-unread"
// broadcastStreamSuffix is the suffix for broadcast stream
broadcastStreamSuffix = "broadcast"
// broadcastFriendsStreamSuffix is the suffix for broadcast friends stream
broadcastFriendsStreamSuffix = "broadcast-friends"
// likeStreamSuffix is the suffix for like stream
likeStreamSuffix = "like"
)
+8 -8
View File
@@ -64,28 +64,28 @@ func parseStreamFilterFromRequest(r *http.Request) (RequestModifiers, error) {
UserID: userID,
}
streamOrder := request.QueryStringParam(r, ParamStreamOrder, "d")
streamOrder := request.QueryStringParam(r, paramStreamOrder, "d")
if streamOrder == "o" {
result.SortDirection = "asc"
}
var err error
result.Streams, err = getStreams(request.QueryStringParamList(r, ParamStreamID), userID)
result.Streams, err = getStreams(request.QueryStringParamList(r, paramStreamID), userID)
if err != nil {
return RequestModifiers{}, err
}
result.ExcludeTargets, err = getStreams(request.QueryStringParamList(r, ParamStreamExcludes), userID)
result.ExcludeTargets, err = getStreams(request.QueryStringParamList(r, paramStreamExcludes), userID)
if err != nil {
return RequestModifiers{}, err
}
result.FilterTargets, err = getStreams(request.QueryStringParamList(r, ParamStreamFilters), userID)
result.FilterTargets, err = getStreams(request.QueryStringParamList(r, paramStreamFilters), userID)
if err != nil {
return RequestModifiers{}, err
}
result.Count = request.QueryIntParam(r, ParamStreamMaxItems, 0)
result.Offset = request.QueryIntParam(r, ParamContinuation, 0)
result.StartTime = request.QueryInt64Param(r, ParamStreamStartTime, int64(0))
result.StopTime = request.QueryInt64Param(r, ParamStreamStopTime, int64(0))
result.Count = request.QueryIntParam(r, paramStreamMaxItems, 0)
result.Offset = request.QueryIntParam(r, paramContinuation, 0)
result.StartTime = request.QueryInt64Param(r, paramStreamStartTime, int64(0))
result.StopTime = request.QueryInt64Param(r, paramStreamStopTime, int64(0))
return result, nil
}
+26 -33
View File
@@ -6,34 +6,36 @@ package googlereader // import "miniflux.app/v2/internal/googlereader"
import (
"fmt"
"net/http"
"miniflux.app/v2/internal/http/response"
)
type login struct {
type loginResponse struct {
SID string `json:"SID,omitempty"`
LSID string `json:"LSID,omitempty"`
Auth string `json:"Auth,omitempty"`
}
func (l login) String() string {
func (l loginResponse) String() string {
return fmt.Sprintf("SID=%s\nLSID=%s\nAuth=%s\n", l.SID, l.LSID, l.Auth)
}
type userInfo struct {
type userInfoResponse struct {
UserID string `json:"userId"`
UserName string `json:"userName"`
UserProfileID string `json:"userProfileId"`
UserEmail string `json:"userEmail"`
}
type subscription struct {
ID string `json:"id"`
Title string `json:"title"`
Categories []subscriptionCategory `json:"categories"`
URL string `json:"url"`
HTMLURL string `json:"htmlUrl"`
IconURL string `json:"iconUrl"`
type subscriptionResponse struct {
ID string `json:"id"`
Title string `json:"title"`
Categories []subscriptionCategoryResponse `json:"categories"`
URL string `json:"url"`
HTMLURL string `json:"htmlUrl"`
IconURL string `json:"iconUrl"`
}
type subscriptionsResponse struct {
Subscriptions []subscriptionResponse `json:"subscriptions"`
}
type quickAddResponse struct {
@@ -43,14 +45,11 @@ type quickAddResponse struct {
StreamName string `json:"streamName,omitempty"`
}
type subscriptionCategory struct {
type subscriptionCategoryResponse struct {
ID string `json:"id"`
Label string `json:"label,omitempty"`
Type string `json:"type,omitempty"`
}
type subscriptionsResponse struct {
Subscriptions []subscription `json:"subscriptions"`
}
type itemRef struct {
ID string `json:"id"`
@@ -64,10 +63,10 @@ type streamIDResponse struct {
}
type tagsResponse struct {
Tags []subscriptionCategory `json:"tags"`
Tags []subscriptionCategoryResponse `json:"tags"`
}
type streamContentItems struct {
type streamContentItemsResponse struct {
Direction string `json:"direction"`
ID string `json:"id"`
Title string `json:"title"`
@@ -118,21 +117,15 @@ type contentItemOrigin struct {
HTMLUrl string `json:"htmlUrl"`
}
// Unauthorized sends a not authorized error to the client.
func Unauthorized(w http.ResponseWriter, r *http.Request) {
builder := response.New(w, r)
builder.WithStatus(http.StatusUnauthorized)
builder.WithHeader("Content-Type", "text/plain")
builder.WithHeader("X-Reader-Google-Bad-Token", "true")
builder.WithBody("Unauthorized")
builder.Write()
func sendUnauthorizedResponse(w http.ResponseWriter) {
w.Header().Set("Content-Type", "text/plain")
w.Header().Set("X-Reader-Google-Bad-Token", "true")
w.WriteHeader(http.StatusUnauthorized)
w.Write([]byte("Unauthorized"))
}
// OK sends a ok response to the client.
func OK(w http.ResponseWriter, r *http.Request) {
builder := response.New(w, r)
builder.WithStatus(http.StatusOK)
builder.WithHeader("Content-Type", "text/plain")
builder.WithBody("OK")
builder.Write()
func sendOkayResponse(w http.ResponseWriter) {
w.Header().Set("Content-Type", "text/plain")
w.WriteHeader(http.StatusOK)
w.Write([]byte("OK"))
}
+16 -16
View File
@@ -72,32 +72,32 @@ func (st StreamType) String() string {
func getStream(streamID string, userID int64) (Stream, error) {
switch {
case strings.HasPrefix(streamID, FeedPrefix):
return Stream{Type: FeedStream, ID: strings.TrimPrefix(streamID, FeedPrefix)}, nil
case strings.HasPrefix(streamID, fmt.Sprintf(UserStreamPrefix, userID)) || strings.HasPrefix(streamID, StreamPrefix):
id := strings.TrimPrefix(streamID, fmt.Sprintf(UserStreamPrefix, userID))
id = strings.TrimPrefix(id, StreamPrefix)
case strings.HasPrefix(streamID, feedPrefix):
return Stream{Type: FeedStream, ID: strings.TrimPrefix(streamID, feedPrefix)}, nil
case strings.HasPrefix(streamID, fmt.Sprintf(userStreamPrefix, userID)), strings.HasPrefix(streamID, streamPrefix):
id := strings.TrimPrefix(streamID, fmt.Sprintf(userStreamPrefix, userID))
id = strings.TrimPrefix(id, streamPrefix)
switch id {
case Read:
case readStreamSuffix:
return Stream{ReadStream, ""}, nil
case Starred:
case starredStreamSuffix:
return Stream{StarredStream, ""}, nil
case ReadingList:
case readingListStreamSuffix:
return Stream{ReadingListStream, ""}, nil
case KeptUnread:
case keptUnreadStreamSuffix:
return Stream{KeptUnreadStream, ""}, nil
case Broadcast:
case broadcastStreamSuffix:
return Stream{BroadcastStream, ""}, nil
case BroadcastFriends:
case broadcastFriendsStreamSuffix:
return Stream{BroadcastFriendsStream, ""}, nil
case Like:
case likeStreamSuffix:
return Stream{LikeStream, ""}, nil
default:
return Stream{NoStream, ""}, fmt.Errorf("googlereader: unknown stream with id: %s", id)
}
case strings.HasPrefix(streamID, fmt.Sprintf(UserLabelPrefix, userID)) || strings.HasPrefix(streamID, LabelPrefix):
id := strings.TrimPrefix(streamID, fmt.Sprintf(UserLabelPrefix, userID))
id = strings.TrimPrefix(id, LabelPrefix)
case strings.HasPrefix(streamID, fmt.Sprintf(userLabelPrefix, userID)), strings.HasPrefix(streamID, labelPrefix):
id := strings.TrimPrefix(streamID, fmt.Sprintf(userLabelPrefix, userID))
id = strings.TrimPrefix(id, labelPrefix)
return Stream{LabelStream, id}, nil
case streamID == "":
return Stream{NoStream, ""}, nil
@@ -107,7 +107,7 @@ func getStream(streamID string, userID int64) (Stream, error) {
}
func getStreams(streamIDs []string, userID int64) ([]Stream, error) {
streams := make([]Stream, 0)
streams := make([]Stream, 0, len(streamIDs))
for _, streamID := range streamIDs {
stream, err := getStream(streamID, userID)
if err != nil {
+16 -4
View File
@@ -18,20 +18,26 @@ const (
// New creates a new cookie.
func New(name, value string, isHTTPS bool, path string) *http.Cookie {
return &http.Cookie{
cookie := &http.Cookie{
Name: name,
Value: value,
Path: basePath(path),
Secure: isHTTPS,
HttpOnly: true,
Expires: time.Now().Add(time.Duration(config.Opts.CleanupRemoveSessionsDays()) * 24 * time.Hour),
SameSite: http.SameSiteLaxMode,
SameSite: http.SameSiteStrictMode,
}
// OAuth doesn't work when cookies are in strict mode.
if config.Opts.OAuth2Provider() != "" {
cookie.SameSite = http.SameSiteLaxMode
}
return cookie
}
// Expired returns an expired cookie.
func Expired(name string, isHTTPS bool, path string) *http.Cookie {
return &http.Cookie{
cookie := &http.Cookie{
Name: name,
Value: "",
Path: basePath(path),
@@ -39,8 +45,14 @@ func Expired(name string, isHTTPS bool, path string) *http.Cookie {
HttpOnly: true,
MaxAge: -1,
Expires: time.Date(1970, 1, 1, 0, 0, 0, 0, time.UTC),
SameSite: http.SameSiteLaxMode,
SameSite: http.SameSiteStrictMode,
}
// OAuth doesn't work when cookies are in strict mode.
if config.Opts.OAuth2Provider() != "" {
cookie.SameSite = http.SameSiteLaxMode
}
return cookie
}
func basePath(path string) string {
+22 -3
View File
@@ -139,14 +139,33 @@ func startUnixSocketServer(server *http.Server, socketFile string) {
}
go func() {
slog.Info("Starting server using a Unix socket", slog.String("socket", socketFile))
if err := server.Serve(listener); err != http.ErrServerClosed {
printErrorAndExit("Unix socket server failed to start on %s: %v", socketFile, err)
certFile := config.Opts.CertFile()
keyFile := config.Opts.CertKeyFile()
if certFile != "" && keyFile != "" {
slog.Info("Starting TLS server using a Unix socket",
slog.String("socket", socketFile),
slog.String("cert_file", certFile),
slog.String("key_file", keyFile),
)
// Ensure HTTPS is marked as true if any listener uses TLS
config.Opts.HTTPS = true
if err := server.ServeTLS(listener, certFile, keyFile); err != http.ErrServerClosed {
printErrorAndExit("TLS Unix socket server failed to start on %s: %v", socketFile, err)
}
} else {
slog.Info("Starting server using a Unix socket", slog.String("socket", socketFile))
if err := server.Serve(listener); err != http.ErrServerClosed {
printErrorAndExit("Unix socket server failed to start on %s: %v", socketFile, err)
}
}
}()
}
func startAutoCertTLSServer(server *http.Server, autoTLSConfig *tls.Config) {
if server.TLSConfig == nil {
server.TLSConfig = &tls.Config{}
}
server.TLSConfig.GetCertificate = autoTLSConfig.GetCertificate
server.TLSConfig.NextProtos = autoTLSConfig.NextProtos
+4 -16
View File
@@ -35,12 +35,9 @@ func (c *Client) CreateBookmark(entryURL, entryTitle string) error {
return fmt.Errorf(`linkwarden: invalid API endpoint: %v`, err)
}
requestBody, err := json.Marshal(&linkwardenBookmark{
Url: entryURL,
Name: "",
Description: "",
Tags: []string{},
Collection: map[string]interface{}{},
requestBody, err := json.Marshal(map[string]string{
"url": entryURL,
"name": entryTitle,
})
if err != nil {
@@ -54,8 +51,7 @@ func (c *Client) CreateBookmark(entryURL, entryTitle string) error {
request.Header.Set("Content-Type", "application/json")
request.Header.Set("User-Agent", "Miniflux/"+version.Version)
request.AddCookie(&http.Cookie{Name: "__Secure-next-auth.session-token", Value: c.apiKey})
request.AddCookie(&http.Cookie{Name: "next-auth.session-token", Value: c.apiKey})
request.Header.Set("Authorization", "Bearer "+c.apiKey)
httpClient := &http.Client{Timeout: defaultClientTimeout}
response, err := httpClient.Do(request)
@@ -70,11 +66,3 @@ func (c *Client) CreateBookmark(entryURL, entryTitle string) error {
return nil
}
type linkwardenBookmark struct {
Url string `json:"url"`
Name string `json:"name"`
Description string `json:"description"`
Tags []string `json:"tags"`
Collection map[string]interface{} `json:"collection"`
}
+3 -16
View File
@@ -9,7 +9,7 @@ import (
"fmt"
)
type translationDict map[string]interface{}
type translationDict map[string]any
type catalog map[string]translationDict
var defaultCatalog = make(catalog, len(AvailableLanguages))
@@ -17,7 +17,7 @@ var defaultCatalog = make(catalog, len(AvailableLanguages))
//go:embed translations/*.json
var translationFiles embed.FS
func GetTranslationDict(language string) (translationDict, error) {
func getTranslationDict(language string) (translationDict, error) {
if _, ok := defaultCatalog[language]; !ok {
var err error
if defaultCatalog[language], err = loadTranslationFile(language); err != nil {
@@ -27,21 +27,8 @@ func GetTranslationDict(language string) (translationDict, error) {
return defaultCatalog[language], nil
}
// LoadCatalogMessages loads and parses all translations encoded in JSON.
func LoadCatalogMessages() error {
var err error
for language := range AvailableLanguages {
defaultCatalog[language], err = loadTranslationFile(language)
if err != nil {
return err
}
}
return nil
}
func loadTranslationFile(language string) (translationDict, error) {
translationFileData, err := translationFiles.ReadFile(fmt.Sprintf("translations/%s.json", language))
translationFileData, err := translationFiles.ReadFile("translations/" + language + ".json")
if err != nil {
return nil, err
}
+5 -2
View File
@@ -33,8 +33,11 @@ func TestParser(t *testing.T) {
}
func TestLoadCatalog(t *testing.T) {
if err := LoadCatalogMessages(); err != nil {
t.Fatal(err)
for language := range AvailableLanguages {
_, err := loadTranslationFile(language)
if err != nil {
t.Fatal(err)
}
}
}
+279
View File
@@ -0,0 +1,279 @@
// SPDX-FileCopyrightText: Copyright The Miniflux Authors. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package locale // import "miniflux.app/v2/internal/locale"
import (
"errors"
"testing"
)
func TestNewLocalizedErrorWrapper(t *testing.T) {
originalErr := errors.New("original error message")
translationKey := "error.test_key"
args := []any{"arg1", 42}
wrapper := NewLocalizedErrorWrapper(originalErr, translationKey, args...)
if wrapper.originalErr != originalErr {
t.Errorf("Expected original error to be %v, got %v", originalErr, wrapper.originalErr)
}
if wrapper.translationKey != translationKey {
t.Errorf("Expected translation key to be %q, got %q", translationKey, wrapper.translationKey)
}
if len(wrapper.translationArgs) != 2 {
t.Errorf("Expected 2 translation args, got %d", len(wrapper.translationArgs))
}
if wrapper.translationArgs[0] != "arg1" || wrapper.translationArgs[1] != 42 {
t.Errorf("Expected translation args [arg1, 42], got %v", wrapper.translationArgs)
}
}
func TestLocalizedErrorWrapper_Error(t *testing.T) {
originalErr := errors.New("original error message")
wrapper := NewLocalizedErrorWrapper(originalErr, "error.test_key")
result := wrapper.Error()
if result != originalErr {
t.Errorf("Expected Error() to return original error %v, got %v", originalErr, result)
}
}
func TestLocalizedErrorWrapper_Translate(t *testing.T) {
// Set up test catalog
defaultCatalog = catalog{
"en_US": translationDict{
"error.test_key": "Error: %s (code: %d)",
},
"fr_FR": translationDict{
"error.test_key": "Erreur : %s (code : %d)",
},
}
originalErr := errors.New("original error")
wrapper := NewLocalizedErrorWrapper(originalErr, "error.test_key", "test message", 404)
// Test English translation
result := wrapper.Translate("en_US")
expected := "Error: test message (code: 404)"
if result != expected {
t.Errorf("Expected English translation %q, got %q", expected, result)
}
// Test French translation
result = wrapper.Translate("fr_FR")
expected = "Erreur : test message (code : 404)"
if result != expected {
t.Errorf("Expected French translation %q, got %q", expected, result)
}
// Test with missing language (should use key as fallback with args applied)
result = wrapper.Translate("invalid_lang")
expected = "error.test_key%!(EXTRA string=test message, int=404)"
if result != expected {
t.Errorf("Expected fallback translation %q, got %q", expected, result)
}
}
func TestLocalizedErrorWrapper_TranslateWithEmptyKey(t *testing.T) {
originalErr := errors.New("original error message")
wrapper := NewLocalizedErrorWrapper(originalErr, "")
result := wrapper.Translate("en_US")
expected := "original error message"
if result != expected {
t.Errorf("Expected original error message %q, got %q", expected, result)
}
}
func TestLocalizedErrorWrapper_TranslateWithNoArgs(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"error.simple": "Simple error message",
},
}
originalErr := errors.New("original error")
wrapper := NewLocalizedErrorWrapper(originalErr, "error.simple")
result := wrapper.Translate("en_US")
expected := "Simple error message"
if result != expected {
t.Errorf("Expected translation %q, got %q", expected, result)
}
}
func TestNewLocalizedError(t *testing.T) {
translationKey := "error.validation"
args := []any{"field1", "invalid"}
localizedErr := NewLocalizedError(translationKey, args...)
if localizedErr.translationKey != translationKey {
t.Errorf("Expected translation key to be %q, got %q", translationKey, localizedErr.translationKey)
}
if len(localizedErr.translationArgs) != 2 {
t.Errorf("Expected 2 translation args, got %d", len(localizedErr.translationArgs))
}
if localizedErr.translationArgs[0] != "field1" || localizedErr.translationArgs[1] != "invalid" {
t.Errorf("Expected translation args [field1, invalid], got %v", localizedErr.translationArgs)
}
}
func TestLocalizedError_String(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"error.validation": "Validation failed for %s: %s",
},
}
localizedErr := NewLocalizedError("error.validation", "username", "too short")
result := localizedErr.String()
expected := "Validation failed for username: too short"
if result != expected {
t.Errorf("Expected String() result %q, got %q", expected, result)
}
}
func TestLocalizedError_StringWithMissingTranslation(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{},
}
localizedErr := NewLocalizedError("error.missing", "arg1")
result := localizedErr.String()
expected := "error.missing%!(EXTRA string=arg1)"
if result != expected {
t.Errorf("Expected String() result %q, got %q", expected, result)
}
}
func TestLocalizedError_Error(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"error.database": "Database connection failed: %s",
},
}
localizedErr := NewLocalizedError("error.database", "timeout")
result := localizedErr.Error()
if result == nil {
t.Error("Expected Error() to return a non-nil error")
}
expected := "Database connection failed: timeout"
if result.Error() != expected {
t.Errorf("Expected Error() message %q, got %q", expected, result.Error())
}
}
func TestLocalizedError_Translate(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"error.permission": "Permission denied for %s",
},
"es_ES": translationDict{
"error.permission": "Permiso denegado para %s",
},
}
localizedErr := NewLocalizedError("error.permission", "admin panel")
// Test English translation
result := localizedErr.Translate("en_US")
expected := "Permission denied for admin panel"
if result != expected {
t.Errorf("Expected English translation %q, got %q", expected, result)
}
// Test Spanish translation
result = localizedErr.Translate("es_ES")
expected = "Permiso denegado para admin panel"
if result != expected {
t.Errorf("Expected Spanish translation %q, got %q", expected, result)
}
// Test with missing language
result = localizedErr.Translate("invalid_lang")
expected = "error.permission%!(EXTRA string=admin panel)"
if result != expected {
t.Errorf("Expected fallback translation %q, got %q", expected, result)
}
}
func TestLocalizedError_TranslateWithNoArgs(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"error.generic": "An error occurred",
},
"de_DE": translationDict{
"error.generic": "Ein Fehler ist aufgetreten",
},
}
localizedErr := NewLocalizedError("error.generic")
// Test English
result := localizedErr.Translate("en_US")
expected := "An error occurred"
if result != expected {
t.Errorf("Expected English translation %q, got %q", expected, result)
}
// Test German
result = localizedErr.Translate("de_DE")
expected = "Ein Fehler ist aufgetreten"
if result != expected {
t.Errorf("Expected German translation %q, got %q", expected, result)
}
}
func TestLocalizedError_TranslateWithComplexArgs(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"error.complex": "Error %d: %s occurred at %s with severity %s",
},
}
localizedErr := NewLocalizedError("error.complex", 500, "Internal Server Error", "2024-01-01", "high")
result := localizedErr.Translate("en_US")
expected := "Error 500: Internal Server Error occurred at 2024-01-01 with severity high"
if result != expected {
t.Errorf("Expected complex translation %q, got %q", expected, result)
}
}
func TestLocalizedErrorWrapper_WithNilError(t *testing.T) {
// This tests edge case behavior - what happens with nil error
wrapper := NewLocalizedErrorWrapper(nil, "error.test")
// Error() should return nil
result := wrapper.Error()
if result != nil {
t.Errorf("Expected Error() to return nil, got %v", result)
}
}
func TestLocalizedError_EmptyKey(t *testing.T) {
localizedErr := NewLocalizedError("")
result := localizedErr.String()
expected := ""
if result != expected {
t.Errorf("Expected empty string for empty key, got %q", result)
}
result = localizedErr.Translate("en_US")
if result != expected {
t.Errorf("Expected empty string for empty key translation, got %q", result)
}
}
+27 -71
View File
@@ -5,16 +5,9 @@ package locale // import "miniflux.app/v2/internal/locale"
// See https://localization-guide.readthedocs.io/en/latest/l10n/pluralforms.html
// And http://www.unicode.org/cldr/charts/29/supplemental/language_plural_rules.html
var pluralForms = map[string]func(n int) int{
// nplurals=2; plural=(n != 1);
"default": func(n int) int {
if n != 1 {
return 1
}
return 0
},
// nplurals=6; plural=(n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 ? 4 : 5);
"ar_AR": func(n int) int {
func getPluralForm(lang string, n int) int {
switch lang {
case "ar_AR":
switch {
case n == 0:
return 0
@@ -26,90 +19,53 @@ var pluralForms = map[string]func(n int) int{
return 3
case n%100 >= 11:
return 4
default:
return 5
}
return 5
},
// nplurals=3; plural=(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2;
"cs_CZ": func(n int) int {
case "cs_CZ":
switch {
case n == 1:
return 0
case n >= 2 && n <= 4:
return 1
default:
return 2
}
return 2
},
// nplurals=2; plural=(n > 1);
"fr_FR": func(n int) int {
if n > 1 {
return 1
}
case "id_ID", "ja_JP":
return 0
},
// nplurals=1; plural=0;
"id_ID": func(n int) int {
return 0
},
// nplurals=1; plural=0;
"ja_JP": func(n int) int {
return 0
},
// nplurals=3; plural=(n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);
"pl_PL": func(n int) int {
case "pl_PL":
switch {
case n == 1:
return 0
case n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20):
return 1
default:
return 2
}
return 2
},
// nplurals=2; plural=(n > 1);
"pt_BR": func(n int) int {
if n > 1 {
return 1
}
return 0
},
// nplurals=3; plural=(n==1 ? 0 : n==0 || (n%100 > 0 && n%100 < 20) ? 1 : 2);
"ro_RO": func(n int) int {
case "ro_RO":
switch {
case n == 1:
return 0
case n == 0 || (n%100 > 0 && n%100 < 20):
return 1
default:
return 2
}
return 2
},
"ru_RU": pluralFormRuSrUa,
// nplurals=2; plural=(n > 1);
"tr_TR": func(n int) int {
case "ru_RU", "uk_UA", "sr_RS":
switch {
case n%10 == 1 && n%100 != 11:
return 0
case n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20):
return 1
default:
return 2
}
case "zh_CN", "zh_TW", "nan_Latn_pehoeji":
return 0
default: // includes fr_FR, pr_BR, tr_TR
if n > 1 {
return 1
}
return 0
},
"uk_UA": pluralFormRuSrUa,
"sr_RS": pluralFormRuSrUa,
// nplurals=1; plural=0;
"zh_CN": func(n int) int {
return 0
},
"zh_TW": func(n int) int {
return 0
},
"nan_Latn_pehoeji": func(n int) int {
return 0
},
}
// nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);
func pluralFormRuSrUa(n int) int {
switch {
case n%10 == 1 && n%100 != 11:
return 0
case n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20):
return 1
}
return 2
}
+156 -48
View File
@@ -7,90 +7,198 @@ import "testing"
func TestPluralRules(t *testing.T) {
scenarios := map[string]map[int]int{
// Default rule (covers fr_FR, pt_BR, tr_TR, and other unlisted languages)
"default": {
1: 0,
2: 1,
5: 1,
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
5: 1, // n > 1
},
// Arabic (ar_AR) - 6 forms
"ar_AR": {
0: 0,
1: 1,
2: 2,
5: 3,
11: 4,
200: 5,
0: 0, // n == 0
1: 1, // n == 1
2: 2, // n == 2
3: 3, // n%100 >= 3 && n%100 <= 10
5: 3, // n%100 >= 3 && n%100 <= 10
10: 3, // n%100 >= 3 && n%100 <= 10
11: 4, // n%100 >= 11
15: 4, // n%100 >= 11
99: 4, // n%100 >= 11
100: 5, // default case (n%100 == 0, doesn't match any condition)
101: 5, // default case (n%100 == 1, but n != 1)
200: 5, // default case
},
// Czech (cs_CZ) - 3 forms
"cs_CZ": {
1: 0,
2: 1,
5: 2,
1: 0, // n == 1
2: 1, // n >= 2 && n <= 4
3: 1, // n >= 2 && n <= 4
4: 1, // n >= 2 && n <= 4
5: 2, // default case
},
// French (fr_FR) - uses default rule
"fr_FR": {
1: 0,
2: 1,
5: 1,
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
5: 1, // n > 1
},
// Indonesian (id_ID) - always form 0
"id_ID": {
1: 0,
5: 0,
0: 0,
1: 0,
5: 0,
100: 0,
},
// Japanese (ja_JP) - always form 0
"ja_JP": {
1: 0,
2: 0,
5: 0,
0: 0,
1: 0,
2: 0,
5: 0,
100: 0,
},
// Polish (pl_PL) - 3 forms
"pl_PL": {
1: 0,
2: 1,
5: 2,
1: 0, // n == 1
2: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20)
3: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20)
4: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20)
5: 2, // default case
10: 2, // default case (n%100 < 10, but n%10 not in 2-4)
11: 2, // default case (n%100 >= 10 and < 20)
12: 2, // default case (n%100 >= 10 and < 20)
22: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 >= 20)
24: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 >= 20)
},
// Portuguese Brazilian (pt_BR) - uses default rule
"pt_BR": {
1: 0,
2: 1,
5: 1,
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
5: 1, // n > 1
},
// Romanian (ro_RO) - 3 forms
"ro_RO": {
1: 0,
2: 1,
5: 1,
0: 1, // n == 0 || (n%100 > 0 && n%100 < 20)
1: 0, // n == 1
2: 1, // n == 0 || (n%100 > 0 && n%100 < 20)
5: 1, // n == 0 || (n%100 > 0 && n%100 < 20)
19: 1, // n == 0 || (n%100 > 0 && n%100 < 20)
20: 2, // default case
21: 2, // default case
100: 2, // default case (n%100 == 0, so condition fails)
101: 1, // n%100 == 1, so n%100 > 0 && n%100 < 20
},
// Russian (ru_RU) - 3 forms
"ru_RU": {
1: 0,
2: 1,
5: 2,
1: 0, // n%10 == 1 && n%100 != 11
2: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20)
3: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20)
4: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20)
5: 2, // default case
11: 2, // n%10 == 1 but n%100 == 11, so default case
12: 2, // default case
21: 0, // n%10 == 1 && n%100 != 11
22: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 >= 20)
},
// Serbian (sr_RS) - same as Russian
"sr_RS": {
1: 0,
2: 1,
5: 2,
1: 0, // n%10 == 1 && n%100 != 11
2: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20)
5: 2, // default case
11: 2, // n%10 == 1 but n%100 == 11, so default case
21: 0, // n%10 == 1 && n%100 != 11
},
// Turkish (tr_TR) - uses default rule
"tr_TR": {
1: 0,
2: 1,
5: 1,
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
5: 1, // n > 1
},
// Ukrainian (uk_UA) - same as Russian
"uk_UA": {
1: 0,
2: 1,
5: 2,
1: 0, // n%10 == 1 && n%100 != 11
2: 1, // n%10 >= 2 && n%10 <= 4 && (n%100 < 10 || n%100 >= 20)
5: 2, // default case
11: 2, // n%10 == 1 but n%100 == 11, so default case
21: 0, // n%10 == 1 && n%100 != 11
},
// Chinese Simplified (zh_CN) - always form 0
"zh_CN": {
1: 0,
5: 0,
0: 0,
1: 0,
5: 0,
100: 0,
},
// Chinese Traditional (zh_TW) - always form 0
"zh_TW": {
1: 0,
5: 0,
0: 0,
1: 0,
5: 0,
100: 0,
},
// Min Nan (nan_Latn_pehoeji) - always form 0
"nan_Latn_pehoeji": {
1: 0,
5: 0,
0: 0,
1: 0,
5: 0,
100: 0,
},
// Additional languages from AvailableLanguages that use default rule
"de_DE": {
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
},
"el_EL": {
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
},
"en_US": {
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
},
"es_ES": {
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
},
"fi_FI": {
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
},
"hi_IN": {
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
},
"it_IT": {
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
},
"nl_NL": {
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
},
// Test a language not in the switch (should use default rule)
"unknown_language": {
0: 0, // n <= 1
1: 0, // n <= 1
2: 1, // n > 1
},
}
for rule, values := range scenarios {
for input, expected := range values {
result := pluralForms[rule](input)
result := getPluralForm(rule, input)
if result != expected {
t.Errorf(`Unexpected result for %q rule, got %d instead of %d for %d as input`, rule, result, expected, input)
}
+15 -23
View File
@@ -10,8 +10,13 @@ type Printer struct {
language string
}
// NewPrinter creates a new Printer instance for the given language.
func NewPrinter(language string) *Printer {
return &Printer{language}
}
func (p *Printer) Print(key string) string {
if dict, err := GetTranslationDict(p.language); err == nil {
if dict, err := getTranslationDict(p.language); err == nil {
if str, ok := dict[key]; ok {
if translation, ok := str.(string); ok {
return translation
@@ -22,15 +27,12 @@ func (p *Printer) Print(key string) string {
}
// Printf is like fmt.Printf, but using language-specific formatting.
func (p *Printer) Printf(key string, args ...interface{}) string {
func (p *Printer) Printf(key string, args ...any) string {
translation := key
if dict, err := GetTranslationDict(p.language); err == nil {
str, found := dict[key]
if found {
var valid bool
translation, valid = str.(string)
if !valid {
if dict, err := getTranslationDict(p.language); err == nil {
if str, ok := dict[key]; ok {
if translation, ok = str.(string); !ok {
translation = key
}
}
@@ -41,7 +43,7 @@ func (p *Printer) Printf(key string, args ...interface{}) string {
// Plural returns the translation of the given key by using the language plural form.
func (p *Printer) Plural(key string, n int, args ...interface{}) string {
dict, err := GetTranslationDict(p.language)
dict, err := getTranslationDict(p.language)
if err != nil {
return key
}
@@ -50,22 +52,17 @@ func (p *Printer) Plural(key string, n int, args ...interface{}) string {
var plurals []string
switch v := choices.(type) {
case []interface{}:
case []string:
plurals = v
case []any:
for _, v := range v {
plurals = append(plurals, fmt.Sprint(v))
}
case []string:
plurals = v
default:
return key
}
pluralForm, found := pluralForms[p.language]
if !found {
pluralForm = pluralForms["default"]
}
index := pluralForm(n)
index := getPluralForm(p.language, n)
if len(plurals) > index {
return fmt.Sprintf(plurals[index], args...)
}
@@ -73,8 +70,3 @@ func (p *Printer) Plural(key string, n int, args ...interface{}) string {
return key
}
// NewPrinter creates a new Printer.
func NewPrinter(language string) *Printer {
return &Printer{language}
}
+260 -10
View File
@@ -5,7 +5,7 @@ package locale // import "miniflux.app/v2/internal/locale"
import "testing"
func TestTranslateWithMissingLanguage(t *testing.T) {
func TestPrintfWithMissingLanguage(t *testing.T) {
defaultCatalog = catalog{}
translation := NewPrinter("invalid").Printf("missing.key")
@@ -14,7 +14,7 @@ func TestTranslateWithMissingLanguage(t *testing.T) {
}
}
func TestTranslateWithMissingKey(t *testing.T) {
func TestPrintfWithMissingKey(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"k": "v",
@@ -27,7 +27,7 @@ func TestTranslateWithMissingKey(t *testing.T) {
}
}
func TestTranslateWithExistingKey(t *testing.T) {
func TestPrintfWithExistingKey(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"auth.username": "Login",
@@ -40,7 +40,7 @@ func TestTranslateWithExistingKey(t *testing.T) {
}
}
func TestTranslateWithExistingKeyAndPlaceholder(t *testing.T) {
func TestPrintfWithExistingKeyAndPlaceholder(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"key": "Test: %s",
@@ -56,7 +56,7 @@ func TestTranslateWithExistingKeyAndPlaceholder(t *testing.T) {
}
}
func TestTranslateWithMissingKeyAndPlaceholder(t *testing.T) {
func TestPrintfWithMissingKeyAndPlaceholder(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"auth.username": "Login",
@@ -72,7 +72,7 @@ func TestTranslateWithMissingKeyAndPlaceholder(t *testing.T) {
}
}
func TestTranslateWithInvalidValue(t *testing.T) {
func TestPrintfWithInvalidValue(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"auth.username": "Login",
@@ -88,7 +88,134 @@ func TestTranslateWithInvalidValue(t *testing.T) {
}
}
func TestTranslatePluralWithDefaultRule(t *testing.T) {
func TestPrintWithMissingLanguage(t *testing.T) {
defaultCatalog = catalog{}
translation := NewPrinter("invalid").Print("missing.key")
if translation != "missing.key" {
t.Errorf(`Wrong translation, got %q`, translation)
}
}
func TestPrintWithMissingKey(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"existing.key": "value",
},
}
translation := NewPrinter("en_US").Print("missing.key")
if translation != "missing.key" {
t.Errorf(`Wrong translation, got %q`, translation)
}
}
func TestPrintWithExistingKey(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"auth.username": "Login",
},
}
translation := NewPrinter("en_US").Print("auth.username")
if translation != "Login" {
t.Errorf(`Wrong translation, got %q`, translation)
}
}
func TestPrintWithDifferentLanguages(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"greeting": "Hello",
},
"fr_FR": translationDict{
"greeting": "Bonjour",
},
"es_ES": translationDict{
"greeting": "Hola",
},
}
tests := []struct {
language string
expected string
}{
{"en_US", "Hello"},
{"fr_FR", "Bonjour"},
{"es_ES", "Hola"},
}
for _, test := range tests {
translation := NewPrinter(test.language).Print("greeting")
if translation != test.expected {
t.Errorf(`Wrong translation for %s, got %q instead of %q`, test.language, translation, test.expected)
}
}
}
func TestPrintWithInvalidTranslationType(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"valid.key": "valid string",
"invalid.key": 12345, // not a string
},
}
printer := NewPrinter("en_US")
// Valid string should work
translation := printer.Print("valid.key")
if translation != "valid string" {
t.Errorf(`Wrong translation for valid key, got %q`, translation)
}
// Invalid type should return the key itself
translation = printer.Print("invalid.key")
if translation != "invalid.key" {
t.Errorf(`Wrong translation for invalid key, got %q`, translation)
}
}
func TestPrintWithNilTranslation(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"nil.key": nil,
},
}
translation := NewPrinter("en_US").Print("nil.key")
if translation != "nil.key" {
t.Errorf(`Wrong translation for nil value, got %q`, translation)
}
}
func TestPrintWithEmptyKey(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"": "empty key translation",
},
}
translation := NewPrinter("en_US").Print("")
if translation != "empty key translation" {
t.Errorf(`Wrong translation for empty key, got %q`, translation)
}
}
func TestPrintWithEmptyTranslation(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"empty.value": "",
},
}
translation := NewPrinter("en_US").Print("empty.value")
if translation != "" {
t.Errorf(`Wrong translation for empty value, got %q`, translation)
}
}
func TestPluralWithDefaultRule(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"number_of_users": []string{"%d user (%s)", "%d users (%s)"},
@@ -112,7 +239,7 @@ func TestTranslatePluralWithDefaultRule(t *testing.T) {
}
}
func TestTranslatePluralWithRussianRule(t *testing.T) {
func TestPluralWithRussianRule(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"time_elapsed.years": []string{"%d year", "%d years"},
@@ -143,7 +270,7 @@ func TestTranslatePluralWithRussianRule(t *testing.T) {
}
}
func TestTranslatePluralWithMissingTranslation(t *testing.T) {
func TestPluralWithMissingTranslation(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"number_of_users": []string{"%d user (%s)", "%d users (%s)"},
@@ -157,7 +284,7 @@ func TestTranslatePluralWithMissingTranslation(t *testing.T) {
}
}
func TestTranslatePluralWithInvalidValues(t *testing.T) {
func TestPluralWithInvalidValues(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"number_of_users": []string{"%d user (%s)", "%d users (%s)"},
@@ -172,3 +299,126 @@ func TestTranslatePluralWithInvalidValues(t *testing.T) {
t.Errorf(`Wrong translation, got %q instead of %q`, translation, expected)
}
}
func TestPluralWithMissingLanguage(t *testing.T) {
defaultCatalog = catalog{}
translation := NewPrinter("invalid_language").Plural("test.key", 2)
expected := "test.key"
if translation != expected {
t.Errorf(`Wrong translation, got %q instead of %q`, translation, expected)
}
}
func TestPluralWithAnySliceType(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"test.key": []any{"%d item", "%d items"},
},
}
printer := NewPrinter("en_US")
translation := printer.Plural("test.key", 1, 1)
expected := "1 item"
if translation != expected {
t.Errorf(`Wrong translation for singular, got %q instead of %q`, translation, expected)
}
translation = printer.Plural("test.key", 2, 2)
expected = "2 items"
if translation != expected {
t.Errorf(`Wrong translation for plural, got %q instead of %q`, translation, expected)
}
}
func TestPluralWithMixedAnySliceTypes(t *testing.T) {
defaultCatalog = catalog{
"en_US": translationDict{
"mixed.key": []any{"single: %s", "multiple: %s", "many: %s"},
},
}
printer := NewPrinter("en_US")
// Test first element (should convert first any element to string)
translation := printer.Plural("mixed.key", 0, "test") // n=0 uses index 0
expected := "single: test"
if translation != expected {
t.Errorf(`Wrong translation for index 0, got %q instead of %q`, translation, expected)
}
// Test second element (should use plural form)
translation = printer.Plural("mixed.key", 2, "items") // plural form for default language
expected = "multiple: items"
if translation != expected {
t.Errorf(`Wrong translation for index 1, got %q instead of %q`, translation, expected)
}
}
func TestPluralWithIndexOutOfBounds(t *testing.T) {
defaultCatalog = catalog{
"test_lang": translationDict{
"limited.key": []string{"only one form"},
},
}
// Force a scenario where getPluralForm might return an index >= len(plurals)
// We'll create a scenario with Czech language rules
defaultCatalog["cs_CZ"] = translationDict{
"limited.key": []string{"one form only"}, // Only one form, but Czech has 3 plural forms
}
printer := NewPrinter("cs_CZ")
// n=5 should return index 2 for Czech, but we only have 1 form (index 0)
translation := printer.Plural("limited.key", 5)
expected := "limited.key"
if translation != expected {
t.Errorf(`Wrong translation for out of bounds index, got %q instead of %q`, translation, expected)
}
}
func TestPluralWithVariousLanguageRules(t *testing.T) {
defaultCatalog = catalog{
"ar_AR": translationDict{
"items": []string{"no items", "one item", "two items", "few items", "many items", "other items"},
},
"pl_PL": translationDict{
"files": []string{"one file", "few files", "many files"},
},
"ja_JP": translationDict{
"photos": []string{"photos"},
},
}
tests := []struct {
language string
key string
n int
expected string
}{
// Arabic tests
{"ar_AR", "items", 0, "no items"},
{"ar_AR", "items", 1, "one item"},
{"ar_AR", "items", 2, "two items"},
{"ar_AR", "items", 5, "few items"}, // n%100 >= 3 && n%100 <= 10
{"ar_AR", "items", 15, "many items"}, // n%100 >= 11
// Polish tests
{"pl_PL", "files", 1, "one file"},
{"pl_PL", "files", 3, "few files"}, // n%10 >= 2 && n%10 <= 4
{"pl_PL", "files", 5, "many files"}, // default case
// Japanese tests (always uses same form)
{"ja_JP", "photos", 1, "photos"},
{"ja_JP", "photos", 10, "photos"},
}
for _, test := range tests {
printer := NewPrinter(test.language)
translation := printer.Plural(test.key, test.n)
if translation != test.expected {
t.Errorf(`Wrong translation for %s with n=%d, got %q instead of %q`,
test.language, test.n, translation, test.expected)
}
}
}
+5 -5
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Linkding-Tags",
"form.integration.linkwarden_activate": "Artikel in Linkwarden speichern",
"form.integration.linkwarden_api_key": "Linkwarden-API-Schlüssel",
"form.integration.linkwarden_endpoint": "Linkwarden-API-Endpunkt",
"form.integration.linkwarden_endpoint": "Linkwarden-Base-URL",
"form.integration.matrix_bot_activate": "Neue Artikel in Matrix übertragen",
"form.integration.matrix_bot_chat_id": "ID des Matrix-Raums",
"form.integration.matrix_bot_password": "Passwort für Matrix-Benutzer",
@@ -508,8 +508,8 @@
"page.keyboard_shortcuts.title": "Tastenkürzel",
"page.keyboard_shortcuts.toggle_bookmark_status": "Lesezeichen hinzufügen/entfernen",
"page.keyboard_shortcuts.toggle_entry_attachments": "Artikelanhänge öffnen/schließen",
"page.keyboard_shortcuts.toggle_read_status_next": "Gewählten Artikel als gelesen/ungelesen markieren, fokus als nächstes",
"page.keyboard_shortcuts.toggle_read_status_prev": "Gewählten Artikel als gelesen/ungelesen markieren, fokus vorherige",
"page.keyboard_shortcuts.toggle_read_status_next": "Gewählten Artikel als gelesen/ungelesen markieren, nächsten auswählen",
"page.keyboard_shortcuts.toggle_read_status_prev": "Gewählten Artikel als gelesen/ungelesen markieren, vorherigen auswählen",
"page.login.google_signin": "Anmeldung mit Google",
"page.login.oidc_signin": "Anmeldung mit %s",
"page.login.title": "Anmeldung",
@@ -530,7 +530,7 @@
"page.sessions.table.actions": "Aktionen",
"page.sessions.table.current_session": "Aktuelle Sitzung",
"page.sessions.table.date": "Datum",
"page.sessions.table.ip": "IP-Addresse",
"page.sessions.table.ip": "IP-Adresse",
"page.sessions.table.user_agent": "Benutzeragent",
"page.sessions.title": "Sitzungen",
"page.settings.link_google_account": "Google-Konto verknüpfen",
@@ -614,4 +614,4 @@
"time_elapsed.yesterday": "gestern",
"tooltip.keyboard_shortcuts": "Tastenkürzel: %s",
"tooltip.logged_user": "Angemeldet als %s"
}
}
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Ετικέτες Linkding",
"form.integration.linkwarden_activate": "Αποθήκευση άρθρων στο Linkwarden",
"form.integration.linkwarden_api_key": "Κλειδί API Linkwarden",
"form.integration.linkwarden_endpoint": "Τελικό σημείο Linkwarden API",
"form.integration.linkwarden_endpoint": "URL βάσης Linkwarden",
"form.integration.matrix_bot_activate": "Μεταφορά νέων άρθρων στο Matrix",
"form.integration.matrix_bot_chat_id": "Αναγνωριστικό της αίθουσας Matrix",
"form.integration.matrix_bot_password": "Κωδικός πρόσβασης για τον χρήστη Matrix",
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Linkding Tags",
"form.integration.linkwarden_activate": "Save entries to Linkwarden",
"form.integration.linkwarden_api_key": "Linkwarden API key",
"form.integration.linkwarden_endpoint": "Linkwarden API Endpoint",
"form.integration.linkwarden_endpoint": "Linkwarden Base URL",
"form.integration.matrix_bot_activate": "Push new entries to Matrix",
"form.integration.matrix_bot_chat_id": "ID of Matrix Room",
"form.integration.matrix_bot_password": "Password for Matrix user",
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Etiquetas de Linkding",
"form.integration.linkwarden_activate": "Enviar artículos a Linkwarden",
"form.integration.linkwarden_api_key": "Clave de API de Linkwarden",
"form.integration.linkwarden_endpoint": "Acceso API de Linkwarden",
"form.integration.linkwarden_endpoint": "URL base de Linkwarden",
"form.integration.matrix_bot_activate": "Transferir nuevos artículos a Matrix",
"form.integration.matrix_bot_chat_id": "ID de la sala de Matrix",
"form.integration.matrix_bot_password": "Contraseña para el usuario de Matrix",
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Linkding Tags",
"form.integration.linkwarden_activate": "Tallenna artikkelit Linkkiin",
"form.integration.linkwarden_api_key": "Linkwarden API-avain",
"form.integration.linkwarden_endpoint": "Linkwarden API-päätepiste",
"form.integration.linkwarden_endpoint": "Linkwarden Base URL",
"form.integration.matrix_bot_activate": "Siirrä uudet artikkelit Matrixiin",
"form.integration.matrix_bot_chat_id": "Matrix-huoneen tunnus",
"form.integration.matrix_bot_password": "Matrix-käyttäjän salasana",
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Libellés",
"form.integration.linkwarden_activate": "Sauvegarder les articles vers Linkwarden",
"form.integration.linkwarden_api_key": "Clé d'API de Linkwarden",
"form.integration.linkwarden_endpoint": "URL de l'API de Linkwarden",
"form.integration.linkwarden_endpoint": "URL de base de Linkwarden",
"form.integration.matrix_bot_activate": "Envoyer les nouveaux articles vers Matrix",
"form.integration.matrix_bot_chat_id": "Identifiant de la salle Matrix",
"form.integration.matrix_bot_password": "Mot de passe de l'utilisateur Matrix",
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Linkding Tags",
"form.integration.linkwarden_activate": "Save entries to Linkwarden",
"form.integration.linkwarden_api_key": "Linkwarden API key",
"form.integration.linkwarden_endpoint": "Linkwarden API Endpoint",
"form.integration.linkwarden_endpoint": "लिंकवर्डन बेस यूआरएलL",
"form.integration.matrix_bot_activate": "नए लेखों को मैट्रिक्स में स्थानांतरित करें",
"form.integration.matrix_bot_chat_id": "मैट्रिक्स रूम की आईडी",
"form.integration.matrix_bot_password": "मैट्रिक्स उपयोगकर्ता के लिए पासवर्ड",
+1 -1
View File
@@ -249,7 +249,7 @@
"form.integration.linkding_tags": "Tanda Linkding",
"form.integration.linkwarden_activate": "Simpan artikel ke Linkwarden",
"form.integration.linkwarden_api_key": "Kunci API Linkwarden",
"form.integration.linkwarden_endpoint": "Titik URL API Linkwarden",
"form.integration.linkwarden_endpoint": "URL Dasar Linkwarden",
"form.integration.matrix_bot_activate": "Kirim entri baru ke Matrix",
"form.integration.matrix_bot_chat_id": "ID Ruang Matrix",
"form.integration.matrix_bot_password": "Kata Sandi Matrix",
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Linkding Tags",
"form.integration.linkwarden_activate": "Salva gli articoli su Linkwarden",
"form.integration.linkwarden_api_key": "API key dell'account Linkwarden",
"form.integration.linkwarden_endpoint": "Endpoint dell'API di Linkwarden",
"form.integration.linkwarden_endpoint": "URL di base di Linkwarden",
"form.integration.matrix_bot_activate": "Trasferimento di nuovi articoli a Matrix",
"form.integration.matrix_bot_chat_id": "ID della stanza Matrix",
"form.integration.matrix_bot_password": "Password per l'utente Matrix",
+1 -1
View File
@@ -249,7 +249,7 @@
"form.integration.linkding_tags": "Linkding Tags",
"form.integration.linkwarden_activate": "Linkwarden に記事を保存する",
"form.integration.linkwarden_api_key": "Linkwarden の API key",
"form.integration.linkwarden_endpoint": "Linkwarden の API Endpoint",
"form.integration.linkwarden_endpoint": "リンクワーデン ベース URL",
"form.integration.matrix_bot_activate": "新しい記事をMatrixに転送する",
"form.integration.matrix_bot_chat_id": "MatrixルームのID",
"form.integration.matrix_bot_password": "Matrixユーザ用パスワード",
@@ -249,7 +249,7 @@
"form.integration.linkding_tags": "Linkding khan-á",
"form.integration.linkwarden_activate": "Pó-chûn siau-sit kàu Linkwarden",
"form.integration.linkwarden_api_key": "Linkwarden API só-sî",
"form.integration.linkwarden_endpoint": "Linkwarden API thâu",
"form.integration.linkwarden_endpoint": "Linkwarden Base URL",
"form.integration.matrix_bot_activate": "Thui-sàng siau-sit kàu Matrix",
"form.integration.matrix_bot_chat_id": "Matrix pâng-keng ID",
"form.integration.matrix_bot_password": "Matrix bi̍t-bé",
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Linkding tags",
"form.integration.linkwarden_activate": "Artikelen opslaan in Linkwarden",
"form.integration.linkwarden_api_key": "Linkwarden API-sleutel",
"form.integration.linkwarden_endpoint": "Linkwarden URL",
"form.integration.linkwarden_endpoint": "Linkwarden Basis URL",
"form.integration.matrix_bot_activate": "Nieuwe artikelen opslaan in Matrix",
"form.integration.matrix_bot_chat_id": "ID van Matrix-kamer",
"form.integration.matrix_bot_password": "Wachtwoord voor Matrix-gebruiker",
+1 -1
View File
@@ -255,7 +255,7 @@
"form.integration.linkding_tags": "Znaczniki Linkding",
"form.integration.linkwarden_activate": "Zapisuj wpisy w Linkwarden",
"form.integration.linkwarden_api_key": "Klucz API do Linkwarden",
"form.integration.linkwarden_endpoint": "Punkt końcowy API Linkwarden",
"form.integration.linkwarden_endpoint": "Podstawowy adres URL Linkwardena",
"form.integration.matrix_bot_activate": "Przesyłaj nowe wpisy do Matrix",
"form.integration.matrix_bot_chat_id": "Identyfikator pokoju Matrix",
"form.integration.matrix_bot_password": "Hasło do Matrix",
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Linkding Tags",
"form.integration.linkwarden_activate": "Salvar itens no Linkwarden",
"form.integration.linkwarden_api_key": "Chave de API do Linkwarden",
"form.integration.linkwarden_endpoint": "Endpoint de API do Linkwarden",
"form.integration.linkwarden_endpoint": "URL base do Linkwarden",
"form.integration.matrix_bot_activate": "Transferir novos artigos para o Matrix",
"form.integration.matrix_bot_chat_id": "Identificação da sala Matrix",
"form.integration.matrix_bot_password": "Palavra-passe para utilizador da Matrix",
+1 -1
View File
@@ -255,7 +255,7 @@
"form.integration.linkding_tags": "TAG-uri Linkding",
"form.integration.linkwarden_activate": "Salvează intrările în Linkwarden",
"form.integration.linkwarden_api_key": "Cheie API Linkwarden",
"form.integration.linkwarden_endpoint": "Endpoint API Linkwarden",
"form.integration.linkwarden_endpoint": "URL-ul de bază Linkwarden",
"form.integration.matrix_bot_activate": "Împinge intrările noi pe Matrix",
"form.integration.matrix_bot_chat_id": "ID-ul Camerei Matrix",
"form.integration.matrix_bot_password": "Parola utilizatorului Matrix",
+1 -1
View File
@@ -255,7 +255,7 @@
"form.integration.linkding_tags": "Теги Linkding",
"form.integration.linkwarden_activate": "Сохранять статьи в Linkwarden",
"form.integration.linkwarden_api_key": "API-ключ Linkwarden",
"form.integration.linkwarden_endpoint": "Конечная точка Linkwarden API",
"form.integration.linkwarden_endpoint": "Базовый URL-адрес Linkwarden",
"form.integration.matrix_bot_activate": "Отправлять статьи в Matrix",
"form.integration.matrix_bot_chat_id": "ID комнаты Matrix",
"form.integration.matrix_bot_password": "Пароль пользователя Matrix",
+1 -1
View File
@@ -252,7 +252,7 @@
"form.integration.linkding_tags": "Linkding Etiketleri",
"form.integration.linkwarden_activate": "Makaleleri Linkwarden'e kaydet",
"form.integration.linkwarden_api_key": "Linkwarden API Anahtarı",
"form.integration.linkwarden_endpoint": "Linkwarden API Uç Noktası",
"form.integration.linkwarden_endpoint": "Linkwarden Temel URL'si",
"form.integration.matrix_bot_activate": "Yeni makaleleri Matrix'e aktarın",
"form.integration.matrix_bot_chat_id": "Matrix odasının kimliği",
"form.integration.matrix_bot_password": "Matrix kullanıcısı için parola",
+1 -1
View File
@@ -255,7 +255,7 @@
"form.integration.linkding_tags": "Linkding Tags",
"form.integration.linkwarden_activate": "Зберігати статті до Linkwarden",
"form.integration.linkwarden_api_key": "Ключ API Linkwarden",
"form.integration.linkwarden_endpoint": "Linkwarden API Endpoint",
"form.integration.linkwarden_endpoint": "Базова URL-адреса Linkwarden",
"form.integration.matrix_bot_activate": "Перенесення нових статей в Матрицю",
"form.integration.matrix_bot_chat_id": "Ідентифікатор кімнати Матриці",
"form.integration.matrix_bot_password": "Пароль для користувача Matrix",
+270 -270
View File
@@ -6,39 +6,39 @@
"action.import": "导入",
"action.login": "登录",
"action.or": "或",
"action.remove": "除",
"action.remove_feed": "除此源",
"action.remove": "除",
"action.remove_feed": "除此订阅源",
"action.save": "保存",
"action.subscribe": "订阅",
"action.update": "更新",
"alert.account_linked": "您的外部账号已关联!",
"alert.account_unlinked": "您的外部帐户已解除关联!",
"alert.background_feed_refresh": "所有订阅源正在后台新。此过程中您仍可继续使用 Miniflux。",
"alert.feed_error": "源存在问题",
"alert.no_bookmark": "目前没有收藏",
"alert.no_category": "目前没有分类",
"alert.no_category_entry": "分类下没有文章",
"alert.no_feed": "目前没有源",
"alert.no_feed_entry": "源中没有文章",
"alert.no_feed_in_category": "没有该类别的源。",
"alert.no_history": "前没有历史",
"alert.no_search_result": "搜索没有结果",
"alert.no_shared_entry": "没有分享文章。",
"alert.no_tag_entry": "没有此标签匹配的条目。",
"alert.no_unread_entry": "目前没有未读文章",
"alert.no_user": "您是目前仅有的用户",
"alert.prefs_saved": "设置已存",
"alert.account_unlinked": "您的外部帐户已解除关联!",
"alert.background_feed_refresh": "所有订阅源正在后台新。您可以在刷新过程中继续使用 Miniflux。",
"alert.feed_error": "此订阅源存在问题",
"alert.no_bookmark": "没有收藏的条目。",
"alert.no_category": "没有分类",
"alert.no_category_entry": "分类下没有条目。",
"alert.no_feed": "你没有任何订阅源。",
"alert.no_feed_entry": "此订阅源中没有条目。",
"alert.no_feed_in_category": "此分类中没有订阅源。",
"alert.no_history": "前没有历史记录。",
"alert.no_search_result": "搜索没有结果",
"alert.no_shared_entry": "没有分享条目。",
"alert.no_tag_entry": "没有匹配此标签的条目。",
"alert.no_unread_entry": "没有未读条目。",
"alert.no_user": "您是唯一的用户",
"alert.prefs_saved": "偏好设置已存!",
"alert.too_many_feeds_refresh": [
"多次触发订阅源更新,请等待 %d 分钟后重试。"
"您触发了太多次订阅源刷新。请在 %d 分钟后重试。"
],
"confirm.loading": "行中…",
"confirm.loading": "行中…",
"confirm.no": "否",
"confirm.question": "您确吗?",
"confirm.question.refresh": "您是否要强制刷新?",
"confirm.question": "您确吗?",
"confirm.question.refresh": "您确定要强制刷新",
"confirm.yes": "是",
"enclosure_media_controls.seek": "查找:",
"enclosure_media_controls.seek": "查找",
"enclosure_media_controls.seek.title": "查找 %s 秒",
"enclosure_media_controls.speed": "速度:",
"enclosure_media_controls.speed": "速度",
"enclosure_media_controls.speed.faster": "快进",
"enclosure_media_controls.speed.faster.title": "速度快进到 %sx",
"enclosure_media_controls.speed.reset": "重置",
@@ -55,112 +55,112 @@
"需要 %d 分钟阅读"
],
"entry.external_link.label": "外部链接",
"entry.save.completed": "完成",
"entry.save.completed": "完成",
"entry.save.label": "保存",
"entry.save.title": "保存这篇文章",
"entry.save.toast.completed": "已保存文章",
"entry.scraper.completed": "抓取完成",
"entry.scraper.label": "抓取全文",
"entry.scraper.title": "抓取全文内容",
"entry.save.title": "保存此条目",
"entry.save.toast.completed": "条目已保存",
"entry.scraper.completed": "完成",
"entry.scraper.label": "下载",
"entry.scraper.title": "获取原始内容",
"entry.share.label": "分享",
"entry.share.title": "分享这篇文章",
"entry.share.title": "分享此条目",
"entry.shared_entry.label": "分享",
"entry.shared_entry.title": "打开公链接",
"entry.state.loading": "载中…",
"entry.shared_entry.title": "打开公链接",
"entry.state.loading": "载中…",
"entry.state.saving": "保存中…",
"entry.status.mark_as_read": "标为已读",
"entry.status.mark_as_unread": "标为未读",
"entry.status.title": "更改状态",
"entry.status.title": "更改条目状态",
"entry.status.toast.read": "已标为已读",
"entry.status.toast.unread": "已标为未读",
"entry.tags.label": "标签:",
"entry.tags.more_tags_label": [
"更多标签 (%d)"
"显示 %d 个更多标签"
],
"entry.unshare.label": "取消分享",
"error.api_key_already_exists": "此 API 密钥已存在。",
"error.bad_credentials": "用户名或密码无效",
"error.category_already_exists": "分类已存在",
"error.category_not_found": "分类不存在或不属于用户。",
"error.bad_credentials": "用户名或密码无效",
"error.category_already_exists": "分类已存在",
"error.category_not_found": "分类不存在或不属于用户。",
"error.database_error": "数据库错误: %v。",
"error.different_passwords": "两次输入的密码不同",
"error.duplicate_fever_username": "Fever 用户名已被占用",
"error.duplicate_googlereader_username": "Google Reader 用户名已被占用",
"error.duplicate_linked_account": "该 Provider 已被关联!",
"error.duplicated_feed": "订阅源已经存在。",
"error.empty_file": "文件为空",
"error.entries_per_page_invalid": "每页的文章数无效。",
"error.feed_already_exists": "此源已存在。",
"error.feed_category_not_found": "此类不存在或不属于用户。",
"error.feed_format_not_detected": "无法解析订阅源格式: %v。",
"error.different_passwords": "密码不一致。",
"error.duplicate_fever_username": "已存在其他用户使用相同的 Fever 用户名!",
"error.duplicate_googlereader_username": "已存在其他用户使用相同的 Google Reader 用户名!",
"error.duplicate_linked_account": "已有人与该提供商关联!",
"error.duplicated_feed": "订阅源已经存在。",
"error.empty_file": "文件为空",
"error.entries_per_page_invalid": "每页的条目数无效。",
"error.feed_already_exists": "此订阅源已存在。",
"error.feed_category_not_found": "此类不存在或不属于用户。",
"error.feed_format_not_detected": "无法解析订阅源格式%v。",
"error.feed_invalid_blocklist_rule": "阻止列表规则无效。",
"error.feed_invalid_keeplist_rule": "保留列表规则无效。",
"error.feed_mandatory_fields": "必须填写网址和分类",
"error.feed_not_found": "订阅源不存在或不属于用户。",
"error.feed_mandatory_fields": "必须填写 URL 和分类",
"error.feed_not_found": "订阅源不存在或不属于用户。",
"error.feed_title_not_empty": "订阅源的标题不能为空。",
"error.feed_url_not_empty": "订阅源的 URL 不能为空。",
"error.fields_mandatory": "必须填写全部信息",
"error.http_bad_gateway": "当前由于错误的网关导致该网站无法访问,问题不在 Miniflux,请稍后重试。",
"error.http_body_read": "无法读取HTTP主体: %v。",
"error.http_client_error": "HTTP 客户端错误r: %v。",
"error.http_empty_response": "HTTP 响应内容为空,该网站可能正在使用机器人保护机制。",
"error.http_empty_response_body": "HTTP 响应主体为空。",
"error.http_forbidden": "该网站被禁止访问网站可能有机器人保护机制?",
"error.http_gateway_timeout": "当前由于网关超时导致该网站无法访问,问题不在 Miniflux,请稍后重试。",
"error.http_internal_server_error": "当前由于服务器错误导致该网站无法访问,问题不在 Miniflux,请稍后重试。",
"error.http_not_authorized": "该网站访问未授权,可能用户名密码错误。",
"error.http_resource_not_found": "请求资源无法找到,请检查 URL。",
"error.http_response_too_large": "HTTP 响应内容过大您可以在全局设置中增加 HTTP 响应大小限制(需服务器重新启动)。",
"error.http_service_unavailable": "当前由于服务器内部错误导致该网站无法访问,问题不在 Miniflux,请稍后重试。",
"error.http_too_many_requests": "Miniflux 对该网站请求过多次数,请稍后重试或改应用配置。",
"error.http_unexpected_status_code": "当前由于意外的 HTTP 状态码%d 导致该网站无法访问,问题不在 Miniflux,请稍后重试。",
"error.invalid_categories_sorting_order": "无效的分类排序",
"error.fields_mandatory": "必须填写全部信息",
"error.http_bad_gateway": "由于网关错误,网站暂不可用。这不是 Miniflux 的问题,请稍后重试。",
"error.http_body_read": "无法读取 HTTP 正文:%v。",
"error.http_client_error": "HTTP 客户端错误%v。",
"error.http_empty_response": "HTTP 响应为空,该网站可能使用了反爬虫机制。",
"error.http_empty_response_body": "HTTP 响应正文为空。",
"error.http_forbidden": "禁止访问网站可能该网站使用了反爬虫机制?",
"error.http_gateway_timeout": "由于网关超时,网站暂不可用。这不是 Miniflux 的问题,请稍后重试。",
"error.http_internal_server_error": "由于服务器错误,网站暂不可用。这不是 Miniflux 的问题,请稍后重试。",
"error.http_not_authorized": "未经授权访问此网站。可能用户名密码错误。",
"error.http_resource_not_found": "未找到请求资源请检查 URL。",
"error.http_response_too_large": "HTTP 响应过大您可以在全局设置中增加 HTTP 响应大小限制(需重启服务器)。",
"error.http_service_unavailable": "由于内部服务器错误,网站暂不可用。这不是 Miniflux 的问题,请稍后重试。",
"error.http_too_many_requests": "Miniflux 向此网站生成了过多请求。请稍后重试或改应用程序配置。",
"error.http_unexpected_status_code": "由于意外的 HTTP 状态码 %d,网站暂不可用。这不是 Miniflux 的问题,请稍后重试。",
"error.invalid_categories_sorting_order": "无效的分类排序顺序。",
"error.invalid_default_home_page": "无效的默认主页!",
"error.invalid_display_mode": "无效的网页应用显示模式。",
"error.invalid_entry_direction": "无效的输入方向。",
"error.invalid_entry_direction": "无效的条目方向。",
"error.invalid_entry_order": "无效的条目排序。",
"error.invalid_feed_proxy_url": "无效的代理 URL。",
"error.invalid_feed_url": "订阅源 URL 无效。",
"error.invalid_gesture_nav": "手势导航无效。",
"error.invalid_feed_url": "无效的订阅源 URL。",
"error.invalid_gesture_nav": "无效的手势导航。",
"error.invalid_language": "无效的语言。",
"error.invalid_site_url": "网站 URL 无效。",
"error.invalid_site_url": "无效的网站 URL。",
"error.invalid_theme": "无效的主题。",
"error.invalid_timezone": "无效的时区。",
"error.network_operation": "Miniflux 无法访问网站由于网络错误: %v。",
"error.network_timeout": "该网站响应过慢,请求超时: %v",
"error.password_min_length": "请至少输入 6 个字符",
"error.network_operation": "由于网络错误,Miniflux 无法访问网站%v。",
"error.network_timeout": "该网站响应过慢,请求超时%v",
"error.password_min_length": "密码长度至少为 6 个字符",
"error.proxy_url_not_empty": "代理 URL 不能为空。",
"error.settings_block_rule_fieldname_invalid": "无效的阻止规则: 规则 #%d 缺少合法的字段名 (可选: %s)",
"error.settings_block_rule_invalid_regex": "无效的阻止规则: 规则 #%d 的模式字符不是合法的正则表达式",
"error.settings_block_rule_regex_required": "无效的阻止规则: 规则 #%d 的模式字符没有提供",
"error.settings_block_rule_separator_required": "无效的阻止规则: 规则 #%d 的模式字符必须用‘=’分开",
"error.settings_invalid_domain_list": "域名列表无效。请提供一个用空格分隔的域名列表。",
"error.settings_keep_rule_fieldname_invalid": "无效的保留规则: 规则 #%d 缺少合法的字段名 (可选: %s)",
"error.settings_keep_rule_invalid_regex": "无效的保留规则: 规则 #%d 的模式字符不是合法的正则表达式",
"error.settings_keep_rule_regex_required": "无效的保留规则: 规则 #%d 的模式字符没有提供",
"error.settings_keep_rule_separator_required": "无效的保留规则: 规则 #%d 的模式字符必须用‘=’分开",
"error.settings_mandatory_fields": "必须填写用户名、主题、语言以及时区",
"error.settings_block_rule_fieldname_invalid": "无效的阻止规则规则 #%d 缺少合法的字段名(可选%s)",
"error.settings_block_rule_invalid_regex": "无效的阻止规则规则 #%d 的模式字符不是合法的正则表达式",
"error.settings_block_rule_regex_required": "无效的阻止规则规则 #%d 的模式字符没有提供",
"error.settings_block_rule_separator_required": "无效的阻止规则规则 #%d 的模式字符必须用‘=’分开",
"error.settings_invalid_domain_list": "无效的域名列表。请提供空格分隔的域名列表。",
"error.settings_keep_rule_fieldname_invalid": "无效的保留规则规则 #%d 缺少合法的字段名(可选%s)",
"error.settings_keep_rule_invalid_regex": "无效的保留规则规则 #%d 的模式字符不是合法的正则表达式",
"error.settings_keep_rule_regex_required": "无效的保留规则规则 #%d 的模式字符没有提供",
"error.settings_keep_rule_separator_required": "无效的保留规则规则 #%d 的模式字符必须用‘=’分开",
"error.settings_mandatory_fields": "必须填写用户名、主题、语言以及时区",
"error.settings_media_playback_rate_range": "播放速度超出范围",
"error.settings_reading_speed_is_positive": "阅读速度必须是正整数。",
"error.site_url_not_empty": "源网站的 URL 不能为空。",
"error.subscription_not_found": "找不到任何源",
"error.title_required": "必须填写标题",
"error.site_url_not_empty": "站点 URL 不能为空。",
"error.subscription_not_found": "无法找到任何订阅源。",
"error.title_required": "必须填写标题",
"error.tls_error": "TLS 错误: %q。如果您愿意的话可以在订阅源设置里关闭 TLS 验证。",
"error.unable_to_create_api_key": "无法创建此 API 密钥。",
"error.unable_to_create_category": "无法建立这个分类",
"error.unable_to_create_user": "无法创建此用户",
"error.unable_to_detect_rssbridge": "无法使用 RSS-Bridge 检测订阅源: %v。",
"error.unable_to_parse_feed": "无法解析订阅源: %v。",
"error.unable_to_update_category": "无法更新分类",
"error.unable_to_update_feed": "无法更新此",
"error.unable_to_update_user": "无法更新此用户",
"error.unable_to_create_category": "无法创建此分类",
"error.unable_to_create_user": "无法创建此用户",
"error.unable_to_detect_rssbridge": "无法使用 RSS-Bridge 检测订阅源%v。",
"error.unable_to_parse_feed": "无法解析订阅源%v。",
"error.unable_to_update_category": "无法更新分类",
"error.unable_to_update_feed": "无法更新此订阅源。",
"error.unable_to_update_user": "无法更新此用户",
"error.unlink_account_without_password": "您必须设置密码,否则您将无法再次登录。",
"error.user_already_exists": "用户已存在",
"error.user_mandatory_fields": "必须填写用户名",
"error.user_already_exists": "用户已存在",
"error.user_mandatory_fields": "必须填写用户名",
"form.api_key.label.description": "API 密钥标签",
"form.category.hide_globally": "隐藏全局未读列表中的文章",
"form.category.hide_globally": "全局未读列表中隐藏条目",
"form.category.label.title": "标题",
"form.feed.fieldset.general": "通用",
"form.feed.fieldset.general": "常规",
"form.feed.fieldset.integration": "第三方服务",
"form.feed.fieldset.network_settings": "网络设置",
"form.feed.fieldset.rules": "规则",
@@ -168,22 +168,22 @@
"form.feed.label.apprise_service_urls": "使用逗号分隔的 Apprise 服务 URL 列表",
"form.feed.label.block_filter_entry_rules": "条目屏蔽规则",
"form.feed.label.blocklist_rules": "基于正则表达式的屏蔽过滤器",
"form.feed.label.category": "类",
"form.feed.label.cookie": "设置 Cookies",
"form.feed.label.crawler": "抓取全文内容",
"form.feed.label.category": "类",
"form.feed.label.cookie": "设置 Cookie",
"form.feed.label.crawler": "获取原始内容",
"form.feed.label.description": "描述",
"form.feed.label.disable_http2": "关闭 HTTP/2 避免记录指纹",
"form.feed.label.disabled": "请勿刷新此",
"form.feed.label.feed_password": "源密码",
"form.feed.label.disable_http2": "禁用 HTTP/2 避免指纹识别",
"form.feed.label.disabled": "刷新此订阅",
"form.feed.label.feed_password": "订阅源密码",
"form.feed.label.feed_url": "订阅源 URL",
"form.feed.label.feed_username": "源用户名",
"form.feed.label.feed_username": "订阅源用户名",
"form.feed.label.fetch_via_proxy": "使用在应用程序级别配置的代理",
"form.feed.label.hide_globally": "隐藏全局未读列表中的文章",
"form.feed.label.hide_globally": "全局未读列表中隐藏条目",
"form.feed.label.ignore_http_cache": "忽略 HTTP 缓存",
"form.feed.label.keep_filter_entry_rules": "条目允许规则",
"form.feed.label.keeplist_rules": "基于正则表达式的保留过滤器",
"form.feed.label.no_media_player": "没有媒体播放器(音频/视频)",
"form.feed.label.ntfy_activate": "推送条目到ntfy",
"form.feed.label.no_media_player": "媒体播放器音频/视频",
"form.feed.label.ntfy_activate": "推送条目到 Ntfy",
"form.feed.label.ntfy_default_priority": "Ntfy 默认优先级",
"form.feed.label.ntfy_high_priority": "Ntfy 高优先级",
"form.feed.label.ntfy_low_priority": "Ntfy 低优先级",
@@ -192,7 +192,7 @@
"form.feed.label.ntfy_priority": "Ntfy 优先级",
"form.feed.label.ntfy_topic": "Ntfy 主题(可选)",
"form.feed.label.proxy_url": "代理 URL",
"form.feed.label.pushover_activate": "将条目推送至 pushover.net",
"form.feed.label.pushover_activate": "推送条目到 Pushover",
"form.feed.label.pushover_default_priority": "Pushover 默认优先级",
"form.feed.label.pushover_high_priority": "Pushover 高优先级",
"form.feed.label.pushover_low_priority": "Pushover 低优先级",
@@ -201,24 +201,24 @@
"form.feed.label.pushover_priority": "Pushover 消息优先级",
"form.feed.label.rewrite_rules": "内容重写规则",
"form.feed.label.scraper_rules": "抓取规则",
"form.feed.label.site_url": "源网站 URL",
"form.feed.label.site_url": "站 URL",
"form.feed.label.title": "标题",
"form.feed.label.urlrewrite_rules": "URL 重写规则",
"form.feed.label.user_agent": "覆盖默认的用户代理",
"form.feed.label.webhook_url": "覆盖 Webhook URL",
"form.import.label.file": "OPML 文件",
"form.import.label.url": "URL",
"form.integration.apprise_activate": "将新文章推送到 Apprise",
"form.integration.apprise_activate": "将新条目推送到 Apprise",
"form.integration.apprise_services_url": "使用逗号分隔的 Apprise 服务 URL 列表",
"form.integration.apprise_url": "Apprise API URL",
"form.integration.betula_activate": "保存文章到 Betula",
"form.integration.betula_activate": "保存条目到 Betula",
"form.integration.betula_token": "Betula 令牌",
"form.integration.betula_url": "Betula 服务端 URL",
"form.integration.cubox_activate": "保存文章到 Cubox",
"form.integration.cubox_activate": "保存条目到 Cubox",
"form.integration.cubox_api_link": "Cubox API 链接",
"form.integration.discord_activate": "将新文章推送到 Discord",
"form.integration.discord_activate": "推送条目到 Discord",
"form.integration.discord_webhook_link": "Discord Webhook 链接",
"form.integration.espial_activate": "保存文章到 Espial",
"form.integration.espial_activate": "保存条目到 Espial",
"form.integration.espial_api_key": "Espial API 密钥",
"form.integration.espial_endpoint": "Espial API 端点",
"form.integration.espial_tags": "Espial 标签",
@@ -227,297 +227,297 @@
"form.integration.fever_password": "Fever 密码",
"form.integration.fever_username": "Fever 用户名",
"form.integration.googlereader_activate": "启用 Google Reader API",
"form.integration.googlereader_endpoint": "Google Reader API 端点:",
"form.integration.googlereader_endpoint": "Google Reader API 端点",
"form.integration.googlereader_password": "Google Reader 密码",
"form.integration.googlereader_username": "Google Reader 用户名",
"form.integration.instapaper_activate": "保存文章到 Instapaper",
"form.integration.instapaper_activate": "保存条目到 Instapaper",
"form.integration.instapaper_password": "Instapaper 密码",
"form.integration.instapaper_username": "Instapaper 用户名",
"form.integration.karakeep_activate": "保存文章到 Karakeep",
"form.integration.karakeep_activate": "保存条目到 Karakeep",
"form.integration.karakeep_api_key": "Karakeep API 密钥",
"form.integration.karakeep_url": "Karakeep API 端点",
"form.integration.linkace_activate": "保存文章到 LinkAce",
"form.integration.linkace_activate": "保存条目到 LinkAce",
"form.integration.linkace_api_key": "LinkAce API 密钥",
"form.integration.linkace_check_disabled": "关闭链接检查",
"form.integration.linkace_endpoint": "LinkAce API URL",
"form.integration.linkace_check_disabled": "禁用链接检查",
"form.integration.linkace_endpoint": "LinkAce API 端点",
"form.integration.linkace_is_private": "将链接标记为私有",
"form.integration.linkace_tags": "LinkAce 标签",
"form.integration.linkding_activate": "保存文章到 Linkding",
"form.integration.linkding_activate": "保存条目到 Linkding",
"form.integration.linkding_api_key": "Linkding API 密钥",
"form.integration.linkding_bookmark": "标记为未读",
"form.integration.linkding_bookmark": "将书签标记为未读",
"form.integration.linkding_endpoint": "Linkding API 端点",
"form.integration.linkding_tags": "Linkding 默认标签",
"form.integration.linkwarden_activate": "保存文章到 Linkwarden",
"form.integration.linkding_tags": "Linkding 标签",
"form.integration.linkwarden_activate": "保存条目到 Linkwarden",
"form.integration.linkwarden_api_key": "Linkwarden API 密钥",
"form.integration.linkwarden_endpoint": "Linkwarden API 端点",
"form.integration.matrix_bot_activate": "将新文章推送到 Matrix",
"form.integration.matrix_bot_chat_id": "Matrix 聊天 ID",
"form.integration.matrix_bot_password": "Matrix Bot 密码",
"form.integration.linkwarden_endpoint": "Linkwarden 基本 URL",
"form.integration.matrix_bot_activate": "推送新条目到 Matrix",
"form.integration.matrix_bot_chat_id": "Matrix 房间 ID",
"form.integration.matrix_bot_password": "Matrix 用户密码",
"form.integration.matrix_bot_url": "Matrix 服务器 URL",
"form.integration.matrix_bot_user": "Matrix Bot 用户名",
"form.integration.notion_activate": "保存文章到 Notion",
"form.integration.notion_page_id": "Notion 页面ID",
"form.integration.notion_token": "Notion 令牌",
"form.integration.ntfy_activate": "推送条目到 ntfy",
"form.integration.ntfy_api_token": "Ntfy API令牌(可选)",
"form.integration.ntfy_icon_url": "Ntfy 图标 URL (可选)",
"form.integration.matrix_bot_user": "Matrix 用户名",
"form.integration.notion_activate": "保存条目到 Notion",
"form.integration.notion_page_id": "Notion 页面 ID",
"form.integration.notion_token": "Notion 密钥令牌",
"form.integration.ntfy_activate": "推送条目到 Ntfy",
"form.integration.ntfy_api_token": "Ntfy API 令牌(可选)",
"form.integration.ntfy_icon_url": "Ntfy 图标 URL(可选)",
"form.integration.ntfy_internal_links": "点击时使用内部链接(可选)",
"form.integration.ntfy_password": "Ntfy 密码(可选)",
"form.integration.ntfy_topic": "Ntfy 主题(預設,如果未在此源中定義",
"form.integration.ntfy_topic": "Ntfy 主题(如果订阅源中未设置则使用默认值",
"form.integration.ntfy_url": "Ntfy URL(可选,默认为 ntfy.sh",
"form.integration.ntfy_username": "Ntfy 用户名(可选)",
"form.integration.nunux_keeper_activate": "保存文章到 Nunux Keeper",
"form.integration.nunux_keeper_activate": "保存条目到 Nunux Keeper",
"form.integration.nunux_keeper_api_key": "Nunux Keeper API 密钥",
"form.integration.nunux_keeper_endpoint": "Nunux Keeper API 端点",
"form.integration.omnivore_activate": "保存文章到 Omnivore",
"form.integration.omnivore_activate": "保存条目到 Omnivore",
"form.integration.omnivore_api_key": "Omnivore API 密钥",
"form.integration.omnivore_url": "Omnivore API 端点",
"form.integration.pinboard_activate": "保存文章到 Pinboard",
"form.integration.pinboard_bookmark": "标记为未读",
"form.integration.pinboard_activate": "保存条目到 Pinboard",
"form.integration.pinboard_bookmark": "将书签标记为未读",
"form.integration.pinboard_tags": "Pinboard 标签",
"form.integration.pinboard_token": "Pinboard API 令牌",
"form.integration.pushover_activate": "将条目推送至 Pushover",
"form.integration.pushover_device": "Pushover 装置(可选)",
"form.integration.pushover_activate": "推送条目到 Pushover",
"form.integration.pushover_device": "Pushover 设备(可选)",
"form.integration.pushover_prefix": "Pushover URL 前缀(可选)",
"form.integration.pushover_token": "Pushover 应用程序 API 令牌",
"form.integration.pushover_token": "Pushover 应用 API 令牌",
"form.integration.pushover_user": "Pushover 用户密钥",
"form.integration.raindrop_activate": "保存文章到 Raindrop",
"form.integration.raindrop_activate": "保存条目到 Raindrop",
"form.integration.raindrop_collection_id": "集合 ID",
"form.integration.raindrop_tags": "Tags (逗号分隔)",
"form.integration.raindrop_token": "(Test) 令牌",
"form.integration.readeck_activate": "保存文章到 Readeck",
"form.integration.raindrop_tags": "标签(逗号分隔)",
"form.integration.raindrop_token": "(测试)令牌",
"form.integration.readeck_activate": "保存条目到 Readeck",
"form.integration.readeck_api_key": "Readeck API 密钥",
"form.integration.readeck_endpoint": "Readeck API 端点",
"form.integration.readeck_labels": "Readeck 默认标签",
"form.integration.readeck_only_url": "仅发送 URL(而不是完整内容)",
"form.integration.readwise_activate": "保存文章到 Readwise Reader",
"form.integration.readeck_labels": "Readeck 标签",
"form.integration.readeck_only_url": "仅发送 URL(而完整内容)",
"form.integration.readwise_activate": "保存条目到 Readwise Reader",
"form.integration.readwise_api_key": "Readwise Reader 访问令牌",
"form.integration.readwise_api_key_link": "获取你的 Readwise 访问令牌",
"form.integration.rssbridge_activate": "添加订阅时检查 RSS-Bridge",
"form.integration.rssbridge_token": "RSS-Bridge 认证令牌",
"form.integration.rssbridge_url": "RSS-Bridge 服务器 URL",
"form.integration.shaarli_activate": "保存文章到 Shaarli",
"form.integration.shaarli_activate": "保存条目到 Shaarli",
"form.integration.shaarli_api_secret": "Shaarli API 密钥",
"form.integration.shaarli_endpoint": "Shaarli URL",
"form.integration.shiori_activate": "保存文章到 Shiori",
"form.integration.shiori_activate": "保存条目到 Shiori",
"form.integration.shiori_endpoint": "Shiori API 端点",
"form.integration.shiori_password": "Shiori 密码",
"form.integration.shiori_username": "Shiori 用户名",
"form.integration.slack_activate": "将新文章推送到 Slack",
"form.integration.slack_activate": "推送条目到 Slack",
"form.integration.slack_webhook_link": "Slack Webhook 链接",
"form.integration.telegram_bot_activate": "将新文章推送到 Telegram",
"form.integration.telegram_bot_disable_buttons": "不展示按钮",
"form.integration.telegram_bot_activate": "推送新条目到 Telegram 聊天",
"form.integration.telegram_bot_disable_buttons": "禁用按钮",
"form.integration.telegram_bot_disable_notification": "禁用通知",
"form.integration.telegram_bot_disable_web_page_preview": "禁用网页预览",
"form.integration.telegram_bot_token": "机器人令牌",
"form.integration.telegram_chat_id": "聊天 ID",
"form.integration.telegram_topic_id": "主题ID",
"form.integration.wallabag_activate": "保存文章到 Wallabag",
"form.integration.telegram_topic_id": "主题 ID",
"form.integration.wallabag_activate": "保存条目到 Wallabag",
"form.integration.wallabag_client_id": "Wallabag 客户端 ID",
"form.integration.wallabag_client_secret": "Wallabag 客户端 密钥",
"form.integration.wallabag_endpoint": "Wallabag 基 URL",
"form.integration.wallabag_only_url": "仅发送 URL(而不是完整内容)",
"form.integration.wallabag_client_secret": "Wallabag 客户端密钥",
"form.integration.wallabag_endpoint": "Wallabag 基 URL",
"form.integration.wallabag_only_url": "仅发送 URL(而完整内容)",
"form.integration.wallabag_password": "Wallabag 密码",
"form.integration.wallabag_username": "Wallabag 用户名",
"form.integration.webhook_activate": "启用 Webhooks",
"form.integration.webhook_secret": "Webhooks 密钥",
"form.integration.webhook_url": "默认 Webhook URL",
"form.integration.webhook_url": "默认 Webhook URL",
"form.prefs.fieldset.application_settings": "应用设置",
"form.prefs.fieldset.authentication_settings": "用户认证设置",
"form.prefs.fieldset.authentication_settings": "认证设置",
"form.prefs.fieldset.global_feed_settings": "全局订阅源设置",
"form.prefs.fieldset.reader_settings": "阅读器设置",
"form.prefs.help.external_font_hosts": "允许外部字体托管的空格分隔列表。例如:\"fonts.gstatic.com fonts.googleapis.com\"。",
"form.prefs.label.always_open_external_links": "打开外部链接阅读文章",
"form.prefs.label.always_open_external_links": "打开外部链接阅读条目",
"form.prefs.label.categories_sorting_order": "分类排序",
"form.prefs.label.cjk_reading_speed": "中文、韩文和日文的阅读速度(每分钟字符数)",
"form.prefs.label.custom_css": "自定义 CSS",
"form.prefs.label.custom_js": "自定义 JavaScript",
"form.prefs.label.default_home_page": "默认主页",
"form.prefs.label.default_reading_speed": "其他语言的阅读速度(每分钟字数)",
"form.prefs.label.display_mode": "渐进式网络应用程序 (PWA) 显示模式",
"form.prefs.label.entries_per_page": "每页文章数",
"form.prefs.label.entry_order": "文章排序依据",
"form.prefs.label.entry_sorting": "文章排序",
"form.prefs.label.entry_swipe": "在触摸屏上启用输入滑动",
"form.prefs.label.external_font_hosts": "外部字体托管",
"form.prefs.label.gesture_nav": "在条目间导航的手势",
"form.prefs.label.display_mode": "渐进式网络应用程序(PWA)显示模式",
"form.prefs.label.entries_per_page": "每页条目数",
"form.prefs.label.entry_order": "条目排序字段",
"form.prefs.label.entry_sorting": "条目排序",
"form.prefs.label.entry_swipe": "在触摸屏上启用条目滑动",
"form.prefs.label.external_font_hosts": "外部字体主机",
"form.prefs.label.gesture_nav": "在条目间导航的手势",
"form.prefs.label.keyboard_shortcuts": "启用键盘快捷键",
"form.prefs.label.language": "语言",
"form.prefs.label.mark_read_manually": "手动标记条目为已读",
"form.prefs.label.mark_read_on_media_completion": "仅当音频/视频播放完成90%%时标记为已读",
"form.prefs.label.mark_read_on_media_completion": "仅当音频/视频播放完成 90%% 时标记为已读",
"form.prefs.label.mark_read_on_view": "查看时自动将条目标记为已读",
"form.prefs.label.mark_read_on_view_or_media_completion": "当浏览时标记条目为已读。对于音频/视频,当播放完成90%%时标记为已读",
"form.prefs.label.mark_read_on_view_or_media_completion": "当浏览时标记条目为已读。对于音频/视频,当播放完成 90%% 时标记为已读",
"form.prefs.label.media_playback_rate": "音频/视频的播放速度",
"form.prefs.label.open_external_links_in_new_tab": "在新标签页中打开外部链接(为链接添加 target=\"_blank\"",
"form.prefs.label.show_reading_time": "显示文章的预计阅读时间",
"form.prefs.label.show_reading_time": "显示条目的预计阅读时间",
"form.prefs.label.theme": "主题",
"form.prefs.label.timezone": "时区",
"form.prefs.select.alphabetical": "字母顺序",
"form.prefs.select.alphabetical": "字母顺序",
"form.prefs.select.browser": "浏览器",
"form.prefs.select.created_time": "文章创建时间",
"form.prefs.select.created_time": "条目创建时间",
"form.prefs.select.fullscreen": "全屏",
"form.prefs.select.minimal_ui": "最小",
"form.prefs.select.none": "没有任何",
"form.prefs.select.older_first": "旧->新",
"form.prefs.select.publish_time": "文章发布时间",
"form.prefs.select.publish_time": "条目发布时间",
"form.prefs.select.recent_first": "新->旧",
"form.prefs.select.standalone": "独立",
"form.prefs.select.swipe": "滑动",
"form.prefs.select.tap": "双击",
"form.prefs.select.unread_count": "未读计数",
"form.submit.loading": "载中…",
"form.submit.loading": "载中…",
"form.submit.saving": "保存中…",
"form.user.label.admin": "管理员",
"form.user.label.confirmation": "再次输入密码",
"form.user.label.confirmation": "确认密码",
"form.user.label.password": "密码",
"form.user.label.username": "用户名",
"menu.about": "关于",
"menu.add_feed": "新增源",
"menu.add_user": "新建用户",
"menu.add_feed": "添加订阅源",
"menu.add_user": "添加用户",
"menu.api_keys": "API 密钥",
"menu.categories": "分类",
"menu.create_api_key": "创建一个新的 API 密钥",
"menu.create_category": "建分类",
"menu.create_api_key": "创建 API 密钥",
"menu.create_category": "建分类",
"menu.edit_category": "编辑",
"menu.edit_feed": "编辑",
"menu.export": "导出",
"menu.feed_entries": "文章",
"menu.feeds": "源",
"menu.flush_history": "清历史",
"menu.history": "历史",
"menu.home_page": "页",
"menu.feed_entries": "条目",
"menu.feeds": "订阅源",
"menu.flush_history": "清历史记录",
"menu.history": "历史记录",
"menu.home_page": "页",
"menu.import": "导入",
"menu.integrations": "集成",
"menu.logout": "登出",
"menu.mark_all_as_read": "全部标为已读",
"menu.mark_page_as_read": "标为已读",
"menu.preferences": "设置",
"menu.refresh_all_feeds": "后台更新全部源",
"menu.refresh_feed": "新",
"menu.mark_page_as_read": "将此页标为已读",
"menu.preferences": "偏好设置",
"menu.refresh_all_feeds": "后台刷新所有订阅源",
"menu.refresh_feed": "新",
"menu.search": "搜索",
"menu.sessions": "会话",
"menu.settings": "设置",
"menu.shared_entries": "已享的文章",
"menu.show_all_entries": "显示所有文章",
"menu.show_only_starred_entries": "仅显示已收藏文章",
"menu.show_only_unread_entries": "仅显示未读文章",
"menu.shared_entries": "已享的条目",
"menu.show_all_entries": "显示所有条目",
"menu.show_only_starred_entries": "仅显示已收藏条目",
"menu.show_only_unread_entries": "仅显示未读条目",
"menu.starred": "收藏",
"menu.title": "菜单",
"menu.unread": "未读",
"menu.users": "用户",
"page.about.author": "作者:",
"page.about.build_date": "构建日期:",
"page.about.credits": "版权",
"page.about.db_usage": "数据库容量",
"page.about.git_commit": "Git提交:",
"page.about.credits": "鸣谢",
"page.about.db_usage": "数据库大小:",
"page.about.git_commit": "Git 提交",
"page.about.global_config_options": "全局配置选项",
"page.about.go_version": "Go 版本",
"page.about.license": "协议",
"page.about.postgres_version": "Postgres 版本",
"page.about.go_version": "Go 版本:",
"page.about.license": "许可证",
"page.about.postgres_version": "Postgres 版本:",
"page.about.title": "关于",
"page.about.version": "版本",
"page.add_feed.choose_feed": "选择一个源",
"page.about.version": "版本:",
"page.add_feed.choose_feed": "选择订阅源",
"page.add_feed.label.url": "URL",
"page.add_feed.legend.advanced_options": "高级选项",
"page.add_feed.no_category": "没有类别,至少需要有一个类别",
"page.add_feed.submit": "查找源",
"page.add_feed.title": "新源",
"page.api_keys.never_used": "没用过",
"page.add_feed.no_category": "没有分类。您必须至少有一个分类。",
"page.add_feed.submit": "查找订阅源",
"page.add_feed.title": "新建订阅源",
"page.api_keys.never_used": "从未使用",
"page.api_keys.table.actions": "操作",
"page.api_keys.table.created_at": "创建日期",
"page.api_keys.table.description": "描述",
"page.api_keys.table.last_used_at": "最后使用",
"page.api_keys.table.token": "令牌",
"page.api_keys.title": "API 密钥",
"page.categories.entries": "查看内容",
"page.categories.entries": "条目",
"page.categories.feed_count": [
"有 %d 个源"
"有 %d 个订阅源"
],
"page.categories.feeds": "查看源",
"page.categories.no_feed": "没有源",
"page.categories.feeds": "订阅源",
"page.categories.no_feed": "无订阅源。",
"page.categories.title": "分类",
"page.categories_count": [
"%d 分类"
"%d 分类"
],
"page.category_label": "分类: %s",
"page.edit_category.title": "编辑分类 : %s",
"page.edit_category.title": "编辑分类%s",
"page.edit_feed.etag_header": "ETag 标题:",
"page.edit_feed.last_check": "最后检查时间:",
"page.edit_feed.last_modified_header": "最后修改的 Header",
"page.edit_feed.last_parsing_error": "最后一次解析错误",
"page.edit_feed.no_header": "无 Header",
"page.edit_feed.title": "编辑源 : %s",
"page.edit_user.title": "编辑用户 : %s",
"page.edit_feed.title": "编辑订阅源: %s",
"page.edit_user.title": "编辑用户: %s",
"page.entry.attachments": "附件",
"page.feeds.error_count": [
"%d 错误"
],
"page.feeds.last_check": "最后检查时间",
"page.feeds.next_check": "下次检查时间",
"page.feeds.read_counter": "已读文章数",
"page.feeds.title": "源",
"page.history.title": "历史",
"page.feeds.last_check": "最后检查:",
"page.feeds.next_check": "下次检查:",
"page.feeds.read_counter": "已读条目数",
"page.feeds.title": "订阅源",
"page.history.title": "历史记录",
"page.import.title": "导入",
"page.integration.bookmarklet": "书签小应用",
"page.integration.bookmarklet.help": "你可以打开这个特殊的书签直接收藏网站",
"page.integration.bookmarklet.instructions": "拖动这个链接到浏览器书签栏",
"page.integration.bookmarklet.name": "收藏 Miniflux",
"page.integration.bookmarklet.help": "此链接允许您通过浏览器书签直接订阅网站",
"page.integration.bookmarklet.instructions": "将此链接拖动到您的书签栏",
"page.integration.bookmarklet.name": "添加到 Miniflux",
"page.integration.miniflux_api": "Miniflux API",
"page.integration.miniflux_api_endpoint": "API 端点",
"page.integration.miniflux_api_password": "密码",
"page.integration.miniflux_api_password_value": "您账的密码",
"page.integration.miniflux_api_password_value": "您账的密码",
"page.integration.miniflux_api_username": "用户名",
"page.integrations.title": "集成",
"page.keyboard_shortcuts.close_modal": "关闭对话窗口",
"page.keyboard_shortcuts.download_content": "抓取全文内容",
"page.keyboard_shortcuts.go_to_bottom_item": "转到底部项目",
"page.keyboard_shortcuts.go_to_categories": "打开分类页面",
"page.keyboard_shortcuts.go_to_feed": "转到源页面",
"page.keyboard_shortcuts.go_to_feeds": "打开源页面",
"page.keyboard_shortcuts.go_to_history": "打开历史页面",
"page.keyboard_shortcuts.go_to_next_item": "下一文章",
"page.keyboard_shortcuts.go_to_next_page": "下一页",
"page.keyboard_shortcuts.go_to_previous_item": "上一文章",
"page.keyboard_shortcuts.go_to_previous_page": "上一页",
"page.keyboard_shortcuts.go_to_search": "将焦点放在搜索表单上",
"page.keyboard_shortcuts.go_to_settings": "打开设置页面",
"page.keyboard_shortcuts.go_to_starred": "打开收藏页面",
"page.keyboard_shortcuts.go_to_top_item": "转到顶部项目",
"page.keyboard_shortcuts.go_to_unread": "打开未读页面",
"page.keyboard_shortcuts.mark_page_as_read": "标记当前页已读",
"page.keyboard_shortcuts.download_content": "下载原始内容",
"page.keyboard_shortcuts.go_to_bottom_item": "转到最后一条",
"page.keyboard_shortcuts.go_to_categories": "转到分类",
"page.keyboard_shortcuts.go_to_feed": "转到订阅源",
"page.keyboard_shortcuts.go_to_feeds": "转到订阅源列表",
"page.keyboard_shortcuts.go_to_history": "转到历史记录",
"page.keyboard_shortcuts.go_to_next_item": "转到下一条目",
"page.keyboard_shortcuts.go_to_next_page": "转到下一页",
"page.keyboard_shortcuts.go_to_previous_item": "转到上一条目",
"page.keyboard_shortcuts.go_to_previous_page": "转到上一页",
"page.keyboard_shortcuts.go_to_search": "聚焦到搜索框",
"page.keyboard_shortcuts.go_to_settings": "转到设置",
"page.keyboard_shortcuts.go_to_starred": "转到收藏",
"page.keyboard_shortcuts.go_to_top_item": "转到第一条",
"page.keyboard_shortcuts.go_to_unread": "转到未读",
"page.keyboard_shortcuts.mark_page_as_read": "标记当前页已读",
"page.keyboard_shortcuts.open_comments": "打开评论链接",
"page.keyboard_shortcuts.open_comments_same_window": "在当前标签页中打开评论链接",
"page.keyboard_shortcuts.open_item": "打开选定的文章",
"page.keyboard_shortcuts.open_item": "打开选定的条目",
"page.keyboard_shortcuts.open_original": "打开原始链接",
"page.keyboard_shortcuts.open_original_same_window": "在当前标签页中打开原始链接",
"page.keyboard_shortcuts.refresh_all_feeds": "在后台新全部源",
"page.keyboard_shortcuts.remove_feed": "除此源",
"page.keyboard_shortcuts.save_article": "保存文章",
"page.keyboard_shortcuts.refresh_all_feeds": "在后台新全部订阅源",
"page.keyboard_shortcuts.remove_feed": "除此订阅源",
"page.keyboard_shortcuts.save_article": "保存条目",
"page.keyboard_shortcuts.scroll_item_to_top": "滚动到顶部",
"page.keyboard_shortcuts.show_keyboard_shortcuts": "显示快捷键帮助",
"page.keyboard_shortcuts.subtitle.actions": "操作",
"page.keyboard_shortcuts.subtitle.items": "文章导航",
"page.keyboard_shortcuts.subtitle.items": "条目导航",
"page.keyboard_shortcuts.subtitle.pages": "页面导航",
"page.keyboard_shortcuts.subtitle.sections": "区导航",
"page.keyboard_shortcuts.title": "快捷键",
"page.keyboard_shortcuts.subtitle.sections": "区导航",
"page.keyboard_shortcuts.title": "键盘快捷键",
"page.keyboard_shortcuts.toggle_bookmark_status": "切换收藏状态",
"page.keyboard_shortcuts.toggle_entry_attachments": "展开/折叠文章附件",
"page.keyboard_shortcuts.toggle_read_status_next": "切换已读/未读状态, 关注下一",
"page.keyboard_shortcuts.toggle_read_status_prev": "切换已读/未读状态, 关注前一个",
"page.keyboard_shortcuts.toggle_entry_attachments": "切换展开/折叠条目附件",
"page.keyboard_shortcuts.toggle_read_status_next": "切换已读/未读状态,并切换到下一",
"page.keyboard_shortcuts.toggle_read_status_prev": "切换已读/未读状态,并切换到上一项",
"page.login.google_signin": "使用 Google 登录",
"page.login.oidc_signin": "使用 %s 登录",
"page.login.title": "登录",
"page.login.webauthn_login": "使用通行密钥登录",
"page.login.webauthn_login.error": "无法使用通行密钥登录",
"page.login.webauthn_login.help": "如果您正在使用安全密钥,请输入您的用户名如果您正在使用通行密钥(可发现凭证),则无需输入。",
"page.login.webauthn_login.help": "如果您正在使用安全密钥,请输入您的用户名如果您正在使用通行密钥(可发现凭证),则无需输入。",
"page.new_api_key.title": "新的 API 密钥",
"page.new_category.title": "新分类",
"page.new_user.title": "新用户",
"page.new_category.title": "新分类",
"page.new_user.title": "新用户",
"page.offline.message": "您已离线",
"page.offline.refresh_page": "尝试刷新页面",
"page.offline.title": "离线模式",
"page.read_entry_count": [
"%d 阅读文章"
"%d 个已读条目"
],
"page.search.title": "搜索结果",
"page.sessions.table.actions": "操作",
@@ -526,42 +526,42 @@
"page.sessions.table.ip": "IP 地址",
"page.sessions.table.user_agent": "用户代理",
"page.sessions.title": "会话",
"page.settings.link_google_account": "关联我的 Google 账",
"page.settings.link_oidc_account": "关联我的 %s 账",
"page.settings.link_google_account": "关联我的 Google 账",
"page.settings.link_oidc_account": "关联我的 %s 账",
"page.settings.title": "设置",
"page.settings.unlink_google_account": "解除 Google 账号关联",
"page.settings.unlink_oidc_account": "解除 %s 账号关联",
"page.settings.webauthn.actions": "操作",
"page.settings.webauthn.added_on": "添加时间",
"page.settings.webauthn.added_on": "添加",
"page.settings.webauthn.delete": [
"删除 %d 个通行密钥"
],
"page.settings.webauthn.last_seen_on": "最后使用时间",
"page.settings.webauthn.last_seen_on": "最后使用",
"page.settings.webauthn.passkey_name": "通行密钥名称",
"page.settings.webauthn.passkeys": "通行密钥列表",
"page.settings.webauthn.passkeys": "通行密钥",
"page.settings.webauthn.register": "注册通行密钥",
"page.settings.webauthn.register.error": "无法注册通行密钥",
"page.shared_entries.title": "已享的文章",
"page.shared_entries.title": "已享的条目",
"page.shared_entries_count": [
"%d 已分享的文章"
"%d 个共享条目"
],
"page.starred.title": "收藏",
"page.starred_entry_count": [
"%d 收藏的文章"
"%d 收藏条目"
],
"page.total_entry_count": [
"%d 文章总数"
"%d 个条目"
],
"page.unread.title": "未读",
"page.unread_entry_count": [
"%d 未读的文章"
"%d 未读条目"
],
"page.users.actions": "操作",
"page.users.admin.no": "否",
"page.users.admin.yes": "是",
"page.users.is_admin": "管理员",
"page.users.last_login": "最后登录时间",
"page.users.never_logged": "从未登录",
"page.users.last_login": "最后登录",
"page.users.never_logged": "从未",
"page.users.title": "用户",
"page.users.username": "用户名",
"page.webauthn_rename.title": "重命名通行密钥",
@@ -571,7 +571,7 @@
"pagination.previous": "上一页",
"search.label": "搜索",
"search.placeholder": "搜索…",
"search.submit": "查找",
"search.submit": "搜索",
"skip_to_content": "跳转至内容",
"time_elapsed.days": [
"%d 天前"
@@ -594,6 +594,6 @@
"%d 年前"
],
"time_elapsed.yesterday": "昨天",
"tooltip.keyboard_shortcuts": "快捷键: %s",
"tooltip.logged_user": "当前登录 %s"
}
"tooltip.keyboard_shortcuts": "键盘快捷键%s",
"tooltip.logged_user": "登录用户:%s"
}
+1 -1
View File
@@ -249,7 +249,7 @@
"form.integration.linkding_tags": "Linkding 標籤",
"form.integration.linkwarden_activate": "儲存文章到 Linkwarden",
"form.integration.linkwarden_api_key": "Linkwarden API 金鑰",
"form.integration.linkwarden_endpoint": "Linkwarden API 端點",
"form.integration.linkwarden_endpoint": "Linkwarden 基本 URL",
"form.integration.matrix_bot_activate": "推送文章到 Matrix",
"form.integration.matrix_bot_chat_id": "Matrix 房間 ID",
"form.integration.matrix_bot_password": "Matrix 密碼",
+142
View File
@@ -589,3 +589,145 @@ func TestProxyFilterVideoPosterOnce(t *testing.T) {
t.Errorf(`Not expected output: got %s`, output)
}
}
func TestShouldProxifyURLWithMimeType(t *testing.T) {
testCases := []struct {
name string
mediaURL string
mediaMimeType string
mediaProxyOption string
mediaProxyResourceTypes []string
expected bool
}{
{
name: "Empty URL should not be proxified",
mediaURL: "",
mediaMimeType: "image/jpeg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"image"},
expected: false,
},
{
name: "Data URL should not be proxified",
mediaURL: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==",
mediaMimeType: "image/png",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"image"},
expected: false,
},
{
name: "HTTP URL with all mode and matching MIME type should be proxified",
mediaURL: "http://example.com/image.jpg",
mediaMimeType: "image/jpeg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"image"},
expected: true,
},
{
name: "HTTPS URL with all mode and matching MIME type should be proxified",
mediaURL: "https://example.com/image.jpg",
mediaMimeType: "image/jpeg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"image"},
expected: true,
},
{
name: "HTTP URL with http-only mode and matching MIME type should be proxified",
mediaURL: "http://example.com/image.jpg",
mediaMimeType: "image/jpeg",
mediaProxyOption: "http-only",
mediaProxyResourceTypes: []string{"image"},
expected: true,
},
{
name: "HTTPS URL with http-only mode should not be proxified",
mediaURL: "https://example.com/image.jpg",
mediaMimeType: "image/jpeg",
mediaProxyOption: "http-only",
mediaProxyResourceTypes: []string{"image"},
expected: false,
},
{
name: "URL with none mode should not be proxified",
mediaURL: "http://example.com/image.jpg",
mediaMimeType: "image/jpeg",
mediaProxyOption: "none",
mediaProxyResourceTypes: []string{"image"},
expected: false,
},
{
name: "URL with matching MIME type should be proxified",
mediaURL: "http://example.com/video.mp4",
mediaMimeType: "video/mp4",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"video"},
expected: true,
},
{
name: "URL with non-matching MIME type should not be proxified",
mediaURL: "http://example.com/video.mp4",
mediaMimeType: "video/mp4",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"image"},
expected: false,
},
{
name: "URL with multiple resource types and matching MIME type should be proxified",
mediaURL: "http://example.com/audio.mp3",
mediaMimeType: "audio/mp3",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"image", "audio", "video"},
expected: true,
},
{
name: "URL with multiple resource types but non-matching MIME type should not be proxified",
mediaURL: "http://example.com/document.pdf",
mediaMimeType: "application/pdf",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"image", "audio", "video"},
expected: false,
},
{
name: "URL with empty resource types should not be proxified",
mediaURL: "http://example.com/image.jpg",
mediaMimeType: "image/jpeg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{},
expected: false,
},
{
name: "URL with partial MIME type match should be proxified",
mediaURL: "http://example.com/image.jpg",
mediaMimeType: "image/jpeg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"image"},
expected: true,
},
{
name: "URL with audio MIME type and audio resource type should be proxified",
mediaURL: "http://example.com/song.ogg",
mediaMimeType: "audio/ogg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio"},
expected: true,
},
{
name: "URL with video MIME type and video resource type should be proxified",
mediaURL: "http://example.com/movie.webm",
mediaMimeType: "video/webm",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"video"},
expected: true,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
result := ShouldProxifyURLWithMimeType(tc.mediaURL, tc.mediaMimeType, tc.mediaProxyOption, tc.mediaProxyResourceTypes)
if result != tc.expected {
t.Errorf("Expected %v, got %v for URL: %s, MIME type: %s, proxy option: %s, resource types: %v",
tc.expected, result, tc.mediaURL, tc.mediaMimeType, tc.mediaProxyOption, tc.mediaProxyResourceTypes)
}
})
}
}
+35 -9
View File
@@ -41,7 +41,7 @@ func genericProxyRewriter(router *mux.Router, proxifyFunction urlProxyRewriter,
case "image":
doc.Find("img, picture source").Each(func(i int, img *goquery.Selection) {
if srcAttrValue, ok := img.Attr("src"); ok {
if shouldProxy(srcAttrValue, proxyOption) {
if shouldProxifyURL(srcAttrValue, proxyOption) {
img.SetAttr("src", proxifyFunction(router, srcAttrValue))
}
}
@@ -54,7 +54,7 @@ func genericProxyRewriter(router *mux.Router, proxifyFunction urlProxyRewriter,
if !slices.Contains(config.Opts.MediaProxyResourceTypes(), "video") {
doc.Find("video").Each(func(i int, video *goquery.Selection) {
if posterAttrValue, ok := video.Attr("poster"); ok {
if shouldProxy(posterAttrValue, proxyOption) {
if shouldProxifyURL(posterAttrValue, proxyOption) {
video.SetAttr("poster", proxifyFunction(router, posterAttrValue))
}
}
@@ -64,7 +64,7 @@ func genericProxyRewriter(router *mux.Router, proxifyFunction urlProxyRewriter,
case "audio":
doc.Find("audio, audio source").Each(func(i int, audio *goquery.Selection) {
if srcAttrValue, ok := audio.Attr("src"); ok {
if shouldProxy(srcAttrValue, proxyOption) {
if shouldProxifyURL(srcAttrValue, proxyOption) {
audio.SetAttr("src", proxifyFunction(router, srcAttrValue))
}
}
@@ -73,13 +73,13 @@ func genericProxyRewriter(router *mux.Router, proxifyFunction urlProxyRewriter,
case "video":
doc.Find("video, video source").Each(func(i int, video *goquery.Selection) {
if srcAttrValue, ok := video.Attr("src"); ok {
if shouldProxy(srcAttrValue, proxyOption) {
if shouldProxifyURL(srcAttrValue, proxyOption) {
video.SetAttr("src", proxifyFunction(router, srcAttrValue))
}
}
if posterAttrValue, ok := video.Attr("poster"); ok {
if shouldProxy(posterAttrValue, proxyOption) {
if shouldProxifyURL(posterAttrValue, proxyOption) {
video.SetAttr("poster", proxifyFunction(router, posterAttrValue))
}
}
@@ -99,7 +99,7 @@ func proxifySourceSet(element *goquery.Selection, router *mux.Router, proxifyFun
imageCandidates := sanitizer.ParseSrcSetAttribute(srcsetAttrValue)
for _, imageCandidate := range imageCandidates {
if shouldProxy(imageCandidate.ImageURL, proxyOption) {
if shouldProxifyURL(imageCandidate.ImageURL, proxyOption) {
imageCandidate.ImageURL = proxifyFunction(router, imageCandidate.ImageURL)
}
}
@@ -107,7 +107,33 @@ func proxifySourceSet(element *goquery.Selection, router *mux.Router, proxifyFun
element.SetAttr("srcset", imageCandidates.String())
}
func shouldProxy(attrValue, proxyOption string) bool {
return !strings.HasPrefix(attrValue, "data:") &&
(proxyOption == "all" || !urllib.IsHTTPS(attrValue))
// shouldProxifyURL checks if the media URL should be proxified based on the media proxy option and URL scheme.
func shouldProxifyURL(mediaURL, mediaProxyOption string) bool {
switch {
case mediaURL == "":
return false
case strings.HasPrefix(mediaURL, "data:"):
return false
case mediaProxyOption == "all":
return true
case mediaProxyOption != "none" && !urllib.IsHTTPS(mediaURL):
return true
default:
return false
}
}
// ShouldProxifyURLWithMimeType checks if the media URL should be proxified based on the media proxy option, URL scheme, and MIME type.
func ShouldProxifyURLWithMimeType(mediaURL, mediaMimeType, mediaProxyOption string, mediaProxyResourceTypes []string) bool {
if !shouldProxifyURL(mediaURL, mediaProxyOption) {
return false
}
for _, mediaType := range mediaProxyResourceTypes {
if strings.HasPrefix(mediaMimeType, mediaType+"/") {
return true
}
}
return false
}
+20 -33
View File
@@ -7,9 +7,8 @@ import (
"strings"
"github.com/gorilla/mux"
"miniflux.app/v2/internal/config"
"miniflux.app/v2/internal/mediaproxy"
"miniflux.app/v2/internal/urllib"
)
// Enclosure represents an attachment.
@@ -45,8 +44,18 @@ func (e *Enclosure) IsVideo() bool {
func (e *Enclosure) IsImage() bool {
mimeType := strings.ToLower(e.MimeType)
if strings.HasPrefix(mimeType, "image/") {
return true
}
mediaURL := strings.ToLower(e.URL)
return strings.HasPrefix(mimeType, "image/") || strings.HasSuffix(mediaURL, ".jpg") || strings.HasSuffix(mediaURL, ".jpeg") || strings.HasSuffix(mediaURL, ".png") || strings.HasSuffix(mediaURL, ".gif")
return strings.HasSuffix(mediaURL, ".jpg") || strings.HasSuffix(mediaURL, ".jpeg") || strings.HasSuffix(mediaURL, ".png") || strings.HasSuffix(mediaURL, ".gif")
}
// ProxifyEnclosureURL modifies the enclosure URL to use the media proxy if necessary.
func (e *Enclosure) ProxifyEnclosureURL(router *mux.Router, mediaProxyOption string, mediaProxyResourceTypes []string) {
if mediaproxy.ShouldProxifyURLWithMimeType(e.URL, e.MimeType, mediaProxyOption, mediaProxyResourceTypes) {
e.URL = mediaproxy.ProxifyAbsoluteURL(router, e.URL)
}
}
// EnclosureList represents a list of attachments.
@@ -55,8 +64,10 @@ type EnclosureList []*Enclosure
// FindMediaPlayerEnclosure returns the first enclosure that can be played by a media player.
func (el EnclosureList) FindMediaPlayerEnclosure() *Enclosure {
for _, enclosure := range el {
if enclosure.URL != "" && strings.Contains(enclosure.MimeType, "audio/") || strings.Contains(enclosure.MimeType, "video/") {
return enclosure
if enclosure.URL != "" {
if enclosure.IsAudio() || enclosure.IsVideo() {
return enclosure
}
}
}
@@ -65,39 +76,15 @@ func (el EnclosureList) FindMediaPlayerEnclosure() *Enclosure {
func (el EnclosureList) ContainsAudioOrVideo() bool {
for _, enclosure := range el {
if strings.Contains(enclosure.MimeType, "audio/") || strings.Contains(enclosure.MimeType, "video/") {
if enclosure.IsAudio() || enclosure.IsVideo() {
return true
}
}
return false
}
func (el EnclosureList) ProxifyEnclosureURL(router *mux.Router) {
proxyOption := config.Opts.MediaProxyMode()
if proxyOption != "none" {
for i := range el {
if urllib.IsHTTPS(el[i].URL) {
for _, mediaType := range config.Opts.MediaProxyResourceTypes() {
if strings.HasPrefix(el[i].MimeType, mediaType+"/") {
el[i].URL = mediaproxy.ProxifyAbsoluteURL(router, el[i].URL)
break
}
}
}
}
}
}
func (e *Enclosure) ProxifyEnclosureURL(router *mux.Router) {
proxyOption := config.Opts.MediaProxyMode()
if proxyOption == "all" || proxyOption != "none" && !urllib.IsHTTPS(e.URL) {
for _, mediaType := range config.Opts.MediaProxyResourceTypes() {
if strings.HasPrefix(e.MimeType, mediaType+"/") {
e.URL = mediaproxy.ProxifyAbsoluteURL(router, e.URL)
break
}
}
func (el EnclosureList) ProxifyEnclosureURL(router *mux.Router, mediaProxyOption string, mediaProxyResourceTypes []string) {
for _, enclosure := range el {
enclosure.ProxifyEnclosureURL(router, mediaProxyOption, mediaProxyResourceTypes)
}
}
+558 -1
View File
@@ -4,7 +4,12 @@
package model
import (
"net/http"
"os"
"testing"
"github.com/gorilla/mux"
"miniflux.app/v2/internal/config"
)
func TestEnclosure_Html5MimeTypeGivesOriginalMimeType(t *testing.T) {
@@ -26,8 +31,560 @@ func TestEnclosure_Html5MimeTypeReplaceStandardM4vByAppleSpecificMimeType(t *tes
// tested at the time of this commit (06/2023) on latest Firefox & Vivaldi on this feed
// https://www.florenceporcel.com/podcast/lfhdu.xml
t.Fatalf(
"HTML5 MimeType must be replaced by 'video/x-m4v' when originally video/m4v to ensure playbacks in brownser. Got '%s'",
"HTML5 MimeType must be replaced by 'video/x-m4v' when originally video/m4v to ensure playbacks in browsers. Got '%s'",
enclosure.Html5MimeType(),
)
}
}
func TestEnclosure_IsAudio(t *testing.T) {
testCases := []struct {
name string
mimeType string
expected bool
}{
{"MP3 audio", "audio/mpeg", true},
{"WAV audio", "audio/wav", true},
{"OGG audio", "audio/ogg", true},
{"Mixed case audio", "Audio/MP3", true},
{"Video file", "video/mp4", false},
{"Image file", "image/jpeg", false},
{"Text file", "text/plain", false},
{"Empty mime type", "", false},
{"Audio with extra info", "audio/mpeg; charset=utf-8", true},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
enclosure := &Enclosure{MimeType: tc.mimeType}
if got := enclosure.IsAudio(); got != tc.expected {
t.Errorf("IsAudio() = %v, want %v for mime type %s", got, tc.expected, tc.mimeType)
}
})
}
}
func TestEnclosure_IsVideo(t *testing.T) {
testCases := []struct {
name string
mimeType string
expected bool
}{
{"MP4 video", "video/mp4", true},
{"AVI video", "video/avi", true},
{"WebM video", "video/webm", true},
{"M4V video", "video/m4v", true},
{"Mixed case video", "Video/MP4", true},
{"Audio file", "audio/mpeg", false},
{"Image file", "image/jpeg", false},
{"Text file", "text/plain", false},
{"Empty mime type", "", false},
{"Video with extra info", "video/mp4; codecs=\"avc1.42E01E\"", true},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
enclosure := &Enclosure{MimeType: tc.mimeType}
if got := enclosure.IsVideo(); got != tc.expected {
t.Errorf("IsVideo() = %v, want %v for mime type %s", got, tc.expected, tc.mimeType)
}
})
}
}
func TestEnclosure_IsImage(t *testing.T) {
testCases := []struct {
name string
mimeType string
url string
expected bool
}{
{"JPEG image by mime", "image/jpeg", "http://example.com/file", true},
{"PNG image by mime", "image/png", "http://example.com/file", true},
{"GIF image by mime", "image/gif", "http://example.com/file", true},
{"Mixed case image mime", "Image/JPEG", "http://example.com/file", true},
{"JPG file extension", "application/octet-stream", "http://example.com/photo.jpg", true},
{"JPEG file extension", "text/plain", "http://example.com/photo.jpeg", true},
{"PNG file extension", "unknown/type", "http://example.com/photo.png", true},
{"GIF file extension", "binary/data", "http://example.com/photo.gif", true},
{"Mixed case extension", "text/plain", "http://example.com/photo.JPG", true},
{"Image mime and extension", "image/jpeg", "http://example.com/photo.jpg", true},
{"Video file", "video/mp4", "http://example.com/video.mp4", false},
{"Audio file", "audio/mpeg", "http://example.com/audio.mp3", false},
{"Text file", "text/plain", "http://example.com/file.txt", false},
{"No extension", "text/plain", "http://example.com/file", false},
{"Other extension", "text/plain", "http://example.com/file.pdf", false},
{"Empty values", "", "", false},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
enclosure := &Enclosure{MimeType: tc.mimeType, URL: tc.url}
if got := enclosure.IsImage(); got != tc.expected {
t.Errorf("IsImage() = %v, want %v for mime type %s and URL %s", got, tc.expected, tc.mimeType, tc.url)
}
})
}
}
func TestEnclosureList_FindMediaPlayerEnclosure(t *testing.T) {
testCases := []struct {
name string
enclosures EnclosureList
expectedNil bool
}{
{
name: "Returns first audio enclosure",
enclosures: EnclosureList{
&Enclosure{URL: "http://example.com/audio.mp3", MimeType: "audio/mpeg"},
&Enclosure{URL: "http://example.com/video.mp4", MimeType: "video/mp4"},
},
expectedNil: false,
},
{
name: "Returns first video enclosure",
enclosures: EnclosureList{
&Enclosure{URL: "http://example.com/video.mp4", MimeType: "video/mp4"},
&Enclosure{URL: "http://example.com/audio.mp3", MimeType: "audio/mpeg"},
},
expectedNil: false,
},
{
name: "Skips image enclosure and returns audio",
enclosures: EnclosureList{
&Enclosure{URL: "http://example.com/image.jpg", MimeType: "image/jpeg"},
&Enclosure{URL: "http://example.com/audio.mp3", MimeType: "audio/mpeg"},
},
expectedNil: false,
},
{
name: "Skips enclosure with empty URL",
enclosures: EnclosureList{
&Enclosure{URL: "", MimeType: "audio/mpeg"},
&Enclosure{URL: "http://example.com/audio.mp3", MimeType: "audio/mpeg"},
},
expectedNil: false,
},
{
name: "Returns nil for no media enclosures",
enclosures: EnclosureList{
&Enclosure{URL: "http://example.com/image.jpg", MimeType: "image/jpeg"},
&Enclosure{URL: "http://example.com/doc.pdf", MimeType: "application/pdf"},
},
expectedNil: true,
},
{
name: "Returns nil for empty list",
enclosures: EnclosureList{},
expectedNil: true,
},
{
name: "Returns nil for all empty URLs",
enclosures: EnclosureList{
&Enclosure{URL: "", MimeType: "audio/mpeg"},
&Enclosure{URL: "", MimeType: "video/mp4"},
},
expectedNil: true,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
result := tc.enclosures.FindMediaPlayerEnclosure()
if tc.expectedNil {
if result != nil {
t.Errorf("FindMediaPlayerEnclosure() = %v, want nil", result)
}
} else {
if result == nil {
t.Errorf("FindMediaPlayerEnclosure() = nil, want non-nil")
} else if !result.IsAudio() && !result.IsVideo() {
t.Errorf("FindMediaPlayerEnclosure() returned non-media enclosure: %s", result.MimeType)
}
}
})
}
}
func TestEnclosureList_ContainsAudioOrVideo(t *testing.T) {
testCases := []struct {
name string
enclosures EnclosureList
expected bool
}{
{
name: "Contains audio",
enclosures: EnclosureList{
&Enclosure{MimeType: "audio/mpeg"},
&Enclosure{MimeType: "image/jpeg"},
},
expected: true,
},
{
name: "Contains video",
enclosures: EnclosureList{
&Enclosure{MimeType: "image/jpeg"},
&Enclosure{MimeType: "video/mp4"},
},
expected: true,
},
{
name: "Contains both audio and video",
enclosures: EnclosureList{
&Enclosure{MimeType: "audio/mpeg"},
&Enclosure{MimeType: "video/mp4"},
},
expected: true,
},
{
name: "Contains only images",
enclosures: EnclosureList{
&Enclosure{MimeType: "image/jpeg"},
&Enclosure{MimeType: "image/png"},
},
expected: false,
},
{
name: "Contains only documents",
enclosures: EnclosureList{
&Enclosure{MimeType: "application/pdf"},
&Enclosure{MimeType: "text/plain"},
},
expected: false,
},
{
name: "Empty list",
enclosures: EnclosureList{},
expected: false,
},
{
name: "Single audio enclosure",
enclosures: EnclosureList{
&Enclosure{MimeType: "audio/wav"},
},
expected: true,
},
{
name: "Single video enclosure",
enclosures: EnclosureList{
&Enclosure{MimeType: "video/webm"},
},
expected: true,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
result := tc.enclosures.ContainsAudioOrVideo()
if result != tc.expected {
t.Errorf("ContainsAudioOrVideo() = %v, want %v", result, tc.expected)
}
})
}
}
func TestEnclosure_ProxifyEnclosureURL(t *testing.T) {
// Initialize config for testing
os.Clearenv()
os.Setenv("BASE_URL", "http://localhost")
os.Setenv("MEDIA_PROXY_PRIVATE_KEY", "test-private-key")
var err error
parser := config.NewParser()
config.Opts, err = parser.ParseEnvironmentVariables()
if err != nil {
t.Fatalf(`Config parsing failure: %v`, err)
}
router := mux.NewRouter()
router.HandleFunc("/proxy/{encodedDigest}/{encodedURL}", func(w http.ResponseWriter, r *http.Request) {}).Name("proxy")
testCases := []struct {
name string
url string
mimeType string
mediaProxyOption string
mediaProxyResourceTypes []string
expectedURLChanged bool
}{
{
name: "HTTP URL with audio type - proxy mode all",
url: "http://example.com/audio.mp3",
mimeType: "audio/mpeg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedURLChanged: true,
},
{
name: "HTTPS URL with video type - proxy mode all",
url: "https://example.com/video.mp4",
mimeType: "video/mp4",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedURLChanged: true,
},
{
name: "HTTP URL with video type - proxy mode http-only",
url: "http://example.com/video.mp4",
mimeType: "video/mp4",
mediaProxyOption: "http-only",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedURLChanged: true,
},
{
name: "HTTPS URL with video type - proxy mode http-only",
url: "https://example.com/video.mp4",
mimeType: "video/mp4",
mediaProxyOption: "http-only",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedURLChanged: false,
},
{
name: "HTTP URL with image type - not in resource types",
url: "http://example.com/image.jpg",
mimeType: "image/jpeg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedURLChanged: false,
},
{
name: "HTTP URL with image type - in resource types",
url: "http://example.com/image.jpg",
mimeType: "image/jpeg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio", "video", "image"},
expectedURLChanged: true,
},
{
name: "HTTP URL - proxy mode none",
url: "http://example.com/audio.mp3",
mimeType: "audio/mpeg",
mediaProxyOption: "none",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedURLChanged: false,
},
{
name: "Empty URL",
url: "",
mimeType: "audio/mpeg",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedURLChanged: false,
},
{
name: "Non-media MIME type",
url: "http://example.com/doc.pdf",
mimeType: "application/pdf",
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedURLChanged: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
enclosure := &Enclosure{
URL: tc.url,
MimeType: tc.mimeType,
}
originalURL := enclosure.URL
// Call the method
enclosure.ProxifyEnclosureURL(router, tc.mediaProxyOption, tc.mediaProxyResourceTypes)
// Check if URL changed as expected
urlChanged := enclosure.URL != originalURL
if urlChanged != tc.expectedURLChanged {
t.Errorf("ProxifyEnclosureURL() URL changed = %v, want %v. Original: %s, New: %s",
urlChanged, tc.expectedURLChanged, originalURL, enclosure.URL)
}
// If URL should have changed, verify it's not empty
if tc.expectedURLChanged && enclosure.URL == "" {
t.Error("ProxifyEnclosureURL() resulted in empty URL when proxification was expected")
}
// If URL shouldn't have changed, verify it's identical
if !tc.expectedURLChanged && enclosure.URL != originalURL {
t.Errorf("ProxifyEnclosureURL() URL changed unexpectedly from %s to %s", originalURL, enclosure.URL)
}
})
}
}
func TestEnclosureList_ProxifyEnclosureURL(t *testing.T) {
// Initialize config for testing
os.Clearenv()
os.Setenv("BASE_URL", "http://localhost")
os.Setenv("MEDIA_PROXY_PRIVATE_KEY", "test-private-key")
var err error
parser := config.NewParser()
config.Opts, err = parser.ParseEnvironmentVariables()
if err != nil {
t.Fatalf(`Config parsing failure: %v`, err)
}
router := mux.NewRouter()
router.HandleFunc("/proxy/{encodedDigest}/{encodedURL}", func(w http.ResponseWriter, r *http.Request) {}).Name("proxy")
testCases := []struct {
name string
enclosures EnclosureList
mediaProxyOption string
mediaProxyResourceTypes []string
expectedChangedCount int
}{
{
name: "Mixed enclosures with all proxy mode",
enclosures: EnclosureList{
&Enclosure{URL: "http://example.com/audio.mp3", MimeType: "audio/mpeg"},
&Enclosure{URL: "https://example.com/video.mp4", MimeType: "video/mp4"},
&Enclosure{URL: "http://example.com/image.jpg", MimeType: "image/jpeg"},
&Enclosure{URL: "http://example.com/doc.pdf", MimeType: "application/pdf"},
},
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedChangedCount: 2, // audio and video should be proxified
},
{
name: "Mixed enclosures with http-only proxy mode",
enclosures: EnclosureList{
&Enclosure{URL: "http://example.com/audio.mp3", MimeType: "audio/mpeg"},
&Enclosure{URL: "https://example.com/video.mp4", MimeType: "video/mp4"},
&Enclosure{URL: "http://example.com/video2.mp4", MimeType: "video/mp4"},
},
mediaProxyOption: "http-only",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedChangedCount: 2, // only HTTP URLs should be proxified
},
{
name: "No media types in resource list",
enclosures: EnclosureList{
&Enclosure{URL: "http://example.com/audio.mp3", MimeType: "audio/mpeg"},
&Enclosure{URL: "http://example.com/video.mp4", MimeType: "video/mp4"},
},
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"image"},
expectedChangedCount: 0, // no matching resource types
},
{
name: "Proxy mode none",
enclosures: EnclosureList{
&Enclosure{URL: "http://example.com/audio.mp3", MimeType: "audio/mpeg"},
&Enclosure{URL: "http://example.com/video.mp4", MimeType: "video/mp4"},
},
mediaProxyOption: "none",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedChangedCount: 0,
},
{
name: "Empty enclosure list",
enclosures: EnclosureList{},
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedChangedCount: 0,
},
{
name: "Enclosures with empty URLs",
enclosures: EnclosureList{
&Enclosure{URL: "", MimeType: "audio/mpeg"},
&Enclosure{URL: "http://example.com/video.mp4", MimeType: "video/mp4"},
},
mediaProxyOption: "all",
mediaProxyResourceTypes: []string{"audio", "video"},
expectedChangedCount: 1, // only the non-empty URL should be processed
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
// Store original URLs
originalURLs := make([]string, len(tc.enclosures))
for i, enclosure := range tc.enclosures {
originalURLs[i] = enclosure.URL
}
// Call the method
tc.enclosures.ProxifyEnclosureURL(router, tc.mediaProxyOption, tc.mediaProxyResourceTypes)
// Count how many URLs actually changed
changedCount := 0
for i, enclosure := range tc.enclosures {
if enclosure.URL != originalURLs[i] {
changedCount++
// Verify that changed URLs are not empty (unless they were empty originally)
if originalURLs[i] != "" && enclosure.URL == "" {
t.Errorf("Enclosure %d: ProxifyEnclosureURL resulted in empty URL", i)
}
}
}
if changedCount != tc.expectedChangedCount {
t.Errorf("ProxifyEnclosureURL() changed %d URLs, want %d", changedCount, tc.expectedChangedCount)
}
})
}
}
func TestEnclosure_ProxifyEnclosureURL_EdgeCases(t *testing.T) {
// Initialize config for testing
os.Clearenv()
os.Setenv("BASE_URL", "http://localhost")
os.Setenv("MEDIA_PROXY_PRIVATE_KEY", "test-private-key")
var err error
parser := config.NewParser()
config.Opts, err = parser.ParseEnvironmentVariables()
if err != nil {
t.Fatalf(`Config parsing failure: %v`, err)
}
router := mux.NewRouter()
router.HandleFunc("/proxy/{encodedDigest}/{encodedURL}", func(w http.ResponseWriter, r *http.Request) {}).Name("proxy")
t.Run("Empty resource types slice", func(t *testing.T) {
enclosure := &Enclosure{
URL: "http://example.com/audio.mp3",
MimeType: "audio/mpeg",
}
originalURL := enclosure.URL
enclosure.ProxifyEnclosureURL(router, "all", []string{})
// With empty resource types, URL should not change
if enclosure.URL != originalURL {
t.Errorf("URL should not change with empty resource types. Original: %s, New: %s", originalURL, enclosure.URL)
}
})
t.Run("Nil resource types slice", func(t *testing.T) {
enclosure := &Enclosure{
URL: "http://example.com/audio.mp3",
MimeType: "audio/mpeg",
}
originalURL := enclosure.URL
enclosure.ProxifyEnclosureURL(router, "all", nil)
// With nil resource types, URL should not change
if enclosure.URL != originalURL {
t.Errorf("URL should not change with nil resource types. Original: %s, New: %s", originalURL, enclosure.URL)
}
})
t.Run("Invalid proxy mode", func(t *testing.T) {
enclosure := &Enclosure{
URL: "http://example.com/audio.mp3",
MimeType: "audio/mpeg",
}
originalURL := enclosure.URL
enclosure.ProxifyEnclosureURL(router, "invalid-mode", []string{"audio"})
// With invalid proxy mode, the function still proxifies non-HTTPS URLs
// because shouldProxifyURL defaults to checking URL scheme
if enclosure.URL == originalURL {
t.Errorf("URL should change for HTTP URL even with invalid proxy mode. Original: %s, New: %s", originalURL, enclosure.URL)
}
})
}
+5 -5
View File
@@ -37,7 +37,6 @@ type Feed struct {
ParsingErrorCount int `json:"parsing_error_count"`
ScraperRules string `json:"scraper_rules"`
RewriteRules string `json:"rewrite_rules"`
Crawler bool `json:"crawler"`
BlocklistRules string `json:"blocklist_rules"`
KeeplistRules string `json:"keeplist_rules"`
BlockFilterEntryRules string `json:"block_filter_entry_rules"`
@@ -54,12 +53,13 @@ type Feed struct {
FetchViaProxy bool `json:"fetch_via_proxy"`
HideGlobally bool `json:"hide_globally"`
DisableHTTP2 bool `json:"disable_http2"`
PushoverEnabled bool `json:"pushover_enabled"`
NtfyEnabled bool `json:"ntfy_enabled"`
Crawler bool `json:"crawler"`
AppriseServiceURLs string `json:"apprise_service_urls"`
WebhookURL string `json:"webhook_url"`
NtfyEnabled bool `json:"ntfy_enabled"`
NtfyPriority int `json:"ntfy_priority"`
NtfyTopic string `json:"ntfy_topic"`
PushoverEnabled bool `json:"pushover_enabled"`
PushoverPriority int `json:"pushover_priority"`
ProxyURL string `json:"proxy_url"`
@@ -164,15 +164,15 @@ type FeedCreationRequest struct {
IgnoreHTTPCache bool `json:"ignore_http_cache"`
AllowSelfSignedCertificates bool `json:"allow_self_signed_certificates"`
FetchViaProxy bool `json:"fetch_via_proxy"`
HideGlobally bool `json:"hide_globally"`
DisableHTTP2 bool `json:"disable_http2"`
ScraperRules string `json:"scraper_rules"`
RewriteRules string `json:"rewrite_rules"`
BlocklistRules string `json:"blocklist_rules"`
KeeplistRules string `json:"keeplist_rules"`
BlockFilterEntryRules string `json:"block_filter_entry_rules"`
KeepFilterEntryRules string `json:"keep_filter_entry_rules"`
HideGlobally bool `json:"hide_globally"`
UrlRewriteRules string `json:"urlrewrite_rules"`
DisableHTTP2 bool `json:"disable_http2"`
ProxyURL string `json:"proxy_url"`
}
+2 -6
View File
@@ -38,23 +38,19 @@ func NewProxyRotator(proxyURLs []string) (*ProxyRotator, error) {
// GetNextProxy returns the next proxy in the rotation.
func (pr *ProxyRotator) GetNextProxy() *url.URL {
pr.mutex.Lock()
defer pr.mutex.Unlock()
if len(pr.proxies) == 0 {
return nil
}
pr.mutex.Lock()
proxy := pr.proxies[pr.currentIndex]
pr.currentIndex = (pr.currentIndex + 1) % len(pr.proxies)
pr.mutex.Unlock()
return proxy
}
// HasProxies checks if there are any proxies available in the rotator.
func (pr *ProxyRotator) HasProxies() bool {
pr.mutex.Lock()
defer pr.mutex.Unlock()
return len(pr.proxies) > 0
}
+2
View File
@@ -137,6 +137,8 @@ func (a *Atom10Adapter) populateEntries(siteURL string) model.Entries {
if len(categories) == 0 {
categories = a.atomFeed.Categories.CategoryNames()
}
// Sort and deduplicate categories.
sort.Strings(categories)
entry.Tags = slices.Compact(categories)
+19 -17
View File
@@ -1761,6 +1761,8 @@ func TestParseItemWithCategories(t *testing.T) {
<updated>2003-12-13T18:30:02Z</updated>
<summary>Some text.</summary>
<category term='ZZZZ' />
<category term='ZZZZ' />
<category term=" " />
<category term='Technology' label='Science' />
</entry>
</feed>`
@@ -1774,16 +1776,13 @@ func TestParseItemWithCategories(t *testing.T) {
t.Fatalf("Incorrect number of tags, got: %d", len(feed.Entries[0].Tags))
}
expected := "Science"
result := feed.Entries[0].Tags[0]
if result != expected {
t.Errorf("Incorrect entry category, got %q instead of %q", result, expected)
}
expected := []string{"Science", "ZZZZ"}
result := feed.Entries[0].Tags
expected = "ZZZZ"
result = feed.Entries[0].Tags[1]
if result != expected {
t.Errorf("Incorrect entry category, got %q instead of %q", result, expected)
for i, tag := range result {
if tag != expected[i] {
t.Errorf("Incorrect entry tag, got %q instead of %q", tag, expected[i])
}
}
}
@@ -1792,9 +1791,10 @@ func TestParseFeedWithCategories(t *testing.T) {
<feed xmlns="http://www.w3.org/2005/Atom">
<title>Example Feed</title>
<link href="http://example.org/"/>
<category term='Test' label='Some Label' />
<category term='Test' label='Some Label' />
<category term='Test' label='Some Label' />
<category term='C term' label='C label' />
<category term='B term' label='B label' />
<category term='B term' label='B label' />
<category term='A term' label='A label' />
<entry>
<link href="http://www.example.org/entries/1" />
<updated>2003-12-13T18:30:02Z</updated>
@@ -1807,14 +1807,16 @@ func TestParseFeedWithCategories(t *testing.T) {
t.Fatal(err)
}
if len(feed.Entries[0].Tags) != 1 {
if len(feed.Entries[0].Tags) != 3 {
t.Fatalf("Incorrect number of tags, got: %d", len(feed.Entries[0].Tags))
}
expected := "Some Label"
result := feed.Entries[0].Tags[0]
if result != expected {
t.Errorf("Incorrect entry category, got %q instead of %q", result, expected)
expected := []string{"A label", "B label", "C label"}
result := feed.Entries[0].Tags
for i, tag := range result {
if tag != expected[i] {
t.Errorf("Incorrect entry tag, got %q instead of %q", tag, expected[i])
}
}
}
+17 -25
View File
@@ -11,6 +11,7 @@ import (
"net"
"net/http"
"net/url"
"slices"
"time"
"miniflux.app/v2/internal/proxyrotator"
@@ -25,8 +26,8 @@ const (
type RequestBuilder struct {
headers http.Header
clientProxyURL *url.URL
useClientProxy bool
clientTimeout int
useClientProxy bool
withoutRedirects bool
ignoreTLSErrors bool
disableHTTP2 bool
@@ -124,38 +125,29 @@ func (r *RequestBuilder) IgnoreTLSErrors(value bool) *RequestBuilder {
}
func (r *RequestBuilder) ExecuteRequest(requestURL string) (*http.Response, error) {
// We get the safe ciphers
ciphers := tls.CipherSuites()
if r.ignoreTLSErrors {
// and the insecure ones if we are ignoring TLS errors. This allows to connect to badly configured servers anyway
ciphers = append(ciphers, tls.InsecureCipherSuites()...)
}
cipherSuites := make([]uint16, 0, len(ciphers))
for _, cipher := range ciphers {
cipherSuites = append(cipherSuites, cipher.ID)
}
transport := &http.Transport{
Proxy: http.ProxyFromEnvironment,
// Setting `DialContext` disables HTTP/2, this option forces the transport to try HTTP/2 regardless.
ForceAttemptHTTP2: true,
DialContext: (&net.Dialer{
// Default is 30s.
Timeout: 10 * time.Second,
// Default is 30s.
KeepAlive: 15 * time.Second,
Timeout: 10 * time.Second, // Default is 30s.
KeepAlive: 15 * time.Second, // Default is 30s.
}).DialContext,
MaxIdleConns: 50, // Default is 100.
IdleConnTimeout: 10 * time.Second, // Default is 90s.
}
// Default is 100.
MaxIdleConns: 50,
// Default is 90s.
IdleConnTimeout: 10 * time.Second,
TLSClientConfig: &tls.Config{
if r.ignoreTLSErrors {
// Add insecure ciphers if we are ignoring TLS errors. This allows to connect to badly configured servers anyway
ciphers := slices.Concat(tls.CipherSuites(), tls.InsecureCipherSuites())
cipherSuites := make([]uint16, 0, len(ciphers))
for _, cipher := range ciphers {
cipherSuites = append(cipherSuites, cipher.ID)
}
transport.TLSClientConfig = &tls.Config{
CipherSuites: cipherSuites,
InsecureSkipVerify: r.ignoreTLSErrors,
},
InsecureSkipVerify: true,
}
}
if r.disableHTTP2 {
+2 -1
View File
@@ -128,6 +128,7 @@ func matchesEntryRegexRules(rules string, entry *model.Entry, feed *model.Feed,
}
func matchesRule(rule string, entry *model.Entry) bool {
rule = strings.TrimSpace(strings.ReplaceAll(rule, "\r\n", ""))
parts := strings.SplitN(rule, "=", 2)
if len(parts) != 2 {
return false
@@ -166,7 +167,7 @@ func logFilterAction(entry *model.Entry, feed *model.Feed, filterRule string, fi
slog.String("feed_url", feed.FeedURL),
slog.String("entry_url", entry.URL),
slog.String("filter_rule", filterRule),
slog.Any("filter_action", filterAction),
slog.String("filter_action", string(filterAction)),
)
}
+9 -2
View File
@@ -52,12 +52,15 @@ func TestBlockingEntries(t *testing.T) {
{&model.Feed{ID: 1, BlockFilterEntryRules: "EntryURL=(?i)example"}, &model.Entry{URL: "https://example.com", Title: "Some Other"}, &model.User{BlockFilterEntryRules: "EntryTitle=(?i)title"}, true}, // Feed rule matches
{&model.Feed{ID: 1, BlockFilterEntryRules: "EntryURL=(?i)example"}, &model.Entry{URL: "https://different.com", Title: "Some Other"}, &model.User{BlockFilterEntryRules: "EntryTitle=(?i)title"}, false}, // Neither rule matches
{&model.Feed{ID: 1, BlockFilterEntryRules: "EntryURL=(?i)example"}, &model.Entry{URL: "https://example.com", Title: "Some Title"}, &model.User{BlockFilterEntryRules: "EntryTitle=(?i)title"}, true}, // Both rules would match
// Test multiple rules with \r\n separators
{&model.Feed{ID: 1, BlockFilterEntryRules: "EntryURL=(?i)example\r\nEntryTitle=(?i)Test"}, &model.Entry{URL: "https://example.com", Title: "Some Example"}, &model.User{}, true},
{&model.Feed{ID: 1, BlockFilterEntryRules: "EntryURL=(?i)example\r\nEntryTitle=(?i)Test"}, &model.Entry{URL: "https://different.com", Title: "Some Test"}, &model.User{}, true},
}
for _, tc := range scenarios {
for index, tc := range scenarios {
result := IsBlockedEntry(tc.feed, tc.entry, tc.user)
if tc.expected != result {
t.Errorf(`Unexpected result, got %v for entry %q`, result, tc.entry.Title)
t.Errorf(`Unexpected result for scenario %d, got %v for entry %q`, index, result, tc.entry.Title)
}
}
}
@@ -113,6 +116,10 @@ func TestAllowEntries(t *testing.T) {
{&model.Feed{ID: 1, KeepFilterEntryRules: "EntryURL=(?i)example"}, &model.Entry{URL: "https://example.com", Title: "Some Other"}, &model.User{KeepFilterEntryRules: "EntryTitle=(?i)title"}, true}, // Feed rule matches
{&model.Feed{ID: 1, KeepFilterEntryRules: "EntryURL=(?i)example"}, &model.Entry{URL: "https://different.com", Title: "Some Other"}, &model.User{KeepFilterEntryRules: "EntryTitle=(?i)title"}, false}, // Neither rule matches
{&model.Feed{ID: 1, KeepFilterEntryRules: "EntryURL=(?i)example"}, &model.Entry{URL: "https://example.com", Title: "Some Title"}, &model.User{KeepFilterEntryRules: "EntryTitle=(?i)title"}, true}, // Both rules would match
// Test multiple rules with \r\n separators
{&model.Feed{ID: 1, KeepFilterEntryRules: "EntryURL=(?i)example\r\nEntryTitle=(?i)Test"}, &model.Entry{URL: "https://example.com", Title: "Some Example"}, &model.User{}, true},
{&model.Feed{ID: 1, KeepFilterEntryRules: "EntryURL=(?i)example\r\nEntryTitle=(?i)Test"}, &model.Entry{URL: "https://different.com", Title: "Some Test"}, &model.User{}, true},
{&model.Feed{ID: 1, KeepFilterEntryRules: "EntryURL=(?i)example\r\nEntryTitle=(?i)Test"}, &model.Entry{URL: "https://different.com", Title: "Some Example"}, &model.User{}, false},
}
for _, tc := range scenarios {
+8 -8
View File
@@ -13,19 +13,19 @@ import (
"miniflux.app/v2/internal/storage"
)
type IconChecker struct {
type iconChecker struct {
store *storage.Storage
feed *model.Feed
}
func NewIconChecker(store *storage.Storage, feed *model.Feed) *IconChecker {
return &IconChecker{
func NewIconChecker(store *storage.Storage, feed *model.Feed) *iconChecker {
return &iconChecker{
store: store,
feed: feed,
}
}
func (c *IconChecker) fetchAndStoreIcon() {
func (c *iconChecker) fetchAndStoreIcon() {
requestBuilder := fetcher.NewRequestBuilder()
requestBuilder.WithUserAgent(c.feed.UserAgent, config.Opts.HTTPClientUserAgent())
requestBuilder.WithCookie(c.feed.Cookie)
@@ -37,8 +37,8 @@ func (c *IconChecker) fetchAndStoreIcon() {
requestBuilder.IgnoreTLSErrors(c.feed.AllowSelfSignedCertificates)
requestBuilder.DisableHTTP2(c.feed.DisableHTTP2)
iconFinder := NewIconFinder(requestBuilder, c.feed.SiteURL, c.feed.IconURL)
if icon, err := iconFinder.FindIcon(); err != nil {
iconFinder := newIconFinder(requestBuilder, c.feed.SiteURL, c.feed.IconURL)
if icon, err := iconFinder.findIcon(); err != nil {
slog.Debug("Unable to find feed icon",
slog.Int64("feed_id", c.feed.ID),
slog.String("website_url", c.feed.SiteURL),
@@ -71,7 +71,7 @@ func (c *IconChecker) fetchAndStoreIcon() {
}
}
func (c *IconChecker) CreateFeedIconIfMissing() {
func (c *iconChecker) CreateFeedIconIfMissing() {
if c.store.HasFeedIcon(c.feed.ID) {
slog.Debug("Feed icon already exists",
slog.Int64("feed_id", c.feed.ID),
@@ -82,6 +82,6 @@ func (c *IconChecker) CreateFeedIconIfMissing() {
c.fetchAndStoreIcon()
}
func (c *IconChecker) UpdateOrCreateFeedIcon() {
func (c *iconChecker) UpdateOrCreateFeedIcon() {
c.fetchAndStoreIcon()
}
+29 -30
View File
@@ -29,28 +29,28 @@ import (
"golang.org/x/image/draw"
)
type IconFinder struct {
type iconFinder struct {
requestBuilder *fetcher.RequestBuilder
websiteURL string
feedIconURL string
}
func NewIconFinder(requestBuilder *fetcher.RequestBuilder, websiteURL, feedIconURL string) *IconFinder {
return &IconFinder{
func newIconFinder(requestBuilder *fetcher.RequestBuilder, websiteURL, feedIconURL string) *iconFinder {
return &iconFinder{
requestBuilder: requestBuilder,
websiteURL: websiteURL,
feedIconURL: feedIconURL,
}
}
func (f *IconFinder) FindIcon() (*model.Icon, error) {
func (f *iconFinder) findIcon() (*model.Icon, error) {
slog.Debug("Begin icon discovery process",
slog.String("website_url", f.websiteURL),
slog.String("feed_icon_url", f.feedIconURL),
)
if f.feedIconURL != "" {
if icon, err := f.FetchFeedIcon(); err != nil {
if icon, err := f.fetchFeedIcon(); err != nil {
slog.Debug("Unable to download icon from feed",
slog.String("website_url", f.websiteURL),
slog.String("feed_icon_url", f.feedIconURL),
@@ -61,7 +61,7 @@ func (f *IconFinder) FindIcon() (*model.Icon, error) {
}
}
if icon, err := f.FetchIconsFromHTMLDocument(); err != nil {
if icon, err := f.fetchIconsFromHTMLDocument(); err != nil {
slog.Debug("Unable to fetch icons from HTML document",
slog.String("website_url", f.websiteURL),
slog.Any("error", err),
@@ -70,10 +70,10 @@ func (f *IconFinder) FindIcon() (*model.Icon, error) {
return icon, nil
}
return f.FetchDefaultIcon()
return f.fetchDefaultIcon()
}
func (f *IconFinder) FetchDefaultIcon() (*model.Icon, error) {
func (f *iconFinder) fetchDefaultIcon() (*model.Icon, error) {
slog.Debug("Fetching default icon",
slog.String("website_url", f.websiteURL),
)
@@ -83,7 +83,7 @@ func (f *IconFinder) FetchDefaultIcon() (*model.Icon, error) {
return nil, fmt.Errorf(`icon: unable to join root URL and path: %w`, err)
}
icon, err := f.DownloadIcon(iconURL)
icon, err := f.downloadIcon(iconURL)
if err != nil {
return nil, err
}
@@ -91,7 +91,7 @@ func (f *IconFinder) FetchDefaultIcon() (*model.Icon, error) {
return icon, nil
}
func (f *IconFinder) FetchFeedIcon() (*model.Icon, error) {
func (f *iconFinder) fetchFeedIcon() (*model.Icon, error) {
slog.Debug("Fetching feed icon",
slog.String("website_url", f.websiteURL),
slog.String("feed_icon_url", f.feedIconURL),
@@ -102,10 +102,10 @@ func (f *IconFinder) FetchFeedIcon() (*model.Icon, error) {
return nil, fmt.Errorf(`icon: unable to convert icon URL to absolute URL: %w`, err)
}
return f.DownloadIcon(iconURL)
return f.downloadIcon(iconURL)
}
func (f *IconFinder) FetchIconsFromHTMLDocument() (*model.Icon, error) {
func (f *iconFinder) fetchIconsFromHTMLDocument() (*model.Icon, error) {
slog.Debug("Searching icons from HTML document",
slog.String("website_url", f.websiteURL),
)
@@ -145,7 +145,7 @@ func (f *IconFinder) FetchIconsFromHTMLDocument() (*model.Icon, error) {
return nil, fmt.Errorf(`icon: unable to convert icon URL to absolute URL: %w`, err)
}
if icon, err := f.DownloadIcon(iconURL); err != nil {
if icon, err := f.downloadIcon(iconURL); err != nil {
slog.Debug("Unable to download icon from HTML document",
slog.String("website_url", f.websiteURL),
slog.String("icon_url", iconURL),
@@ -163,7 +163,7 @@ func (f *IconFinder) FetchIconsFromHTMLDocument() (*model.Icon, error) {
return nil, nil
}
func (f *IconFinder) DownloadIcon(iconURL string) (*model.Icon, error) {
func (f *iconFinder) downloadIcon(iconURL string) (*model.Icon, error) {
slog.Debug("Downloading icon",
slog.String("website_url", f.websiteURL),
slog.String("icon_url", iconURL),
@@ -241,13 +241,6 @@ func resizeIcon(icon *model.Icon) *model.Icon {
}
func findIconURLsFromHTMLDocument(body io.Reader, contentType string) ([]string, error) {
queries := []string{
"link[rel='icon' i]",
"link[rel='shortcut icon' i]",
"link[rel='icon shortcut' i]",
"link[rel='apple-touch-icon-precomposed.png']",
}
htmlDocumentReader, err := encoding.NewCharsetReader(body, contentType)
if err != nil {
return nil, fmt.Errorf("icon: unable to create charset reader: %w", err)
@@ -258,20 +251,26 @@ func findIconURLsFromHTMLDocument(body io.Reader, contentType string) ([]string,
return nil, fmt.Errorf("icon: unable to read document: %v", err)
}
queries := []string{
"link[rel='icon' i][href]",
"link[rel='shortcut icon' i][href]",
"link[rel='icon shortcut' i][href]",
"link[rel='apple-touch-icon-precomposed.png'][href]",
}
var iconURLs []string
for _, query := range queries {
slog.Debug("Searching icon URL in HTML document", slog.String("query", query))
doc.Find(query).Each(func(i int, s *goquery.Selection) {
if href, exists := s.Attr("href"); exists {
if iconURL := strings.TrimSpace(href); iconURL != "" {
iconURLs = append(iconURLs, iconURL)
slog.Debug("Found icon URL in HTML document",
slog.String("query", query),
slog.String("icon_url", iconURL))
}
for _, s := range doc.Find(query).EachIter() {
href, _ := s.Attr("href")
if iconURL := strings.TrimSpace(href); iconURL != "" {
iconURLs = append(iconURLs, iconURL)
slog.Debug("Found icon URL in HTML document",
slog.String("query", query),
slog.String("icon_url", iconURL))
}
})
}
}
return iconURLs, nil
+1 -1
View File
@@ -115,7 +115,7 @@ func TestParseInvalidImageDataURLWithWrongPrefix(t *testing.T) {
func TestParseDocumentWithWhitespaceIconURL(t *testing.T) {
html := `<link rel="shortcut icon" href="
/static/img/favicon.ico
">`
"><link rel='shortcut icon'><link rel='shortcut icon' href=" ">`
iconURLs, err := findIconURLsFromHTMLDocument(strings.NewReader(html), "text/html")
if err != nil {
+4
View File
@@ -157,6 +157,10 @@ func (j *JSONAdapter) BuildFeed(baseURL string) *model.Feed {
}
}
// Sort and deduplicate tags.
slices.Sort(entry.Tags)
entry.Tags = slices.Compact(entry.Tags)
// Generate a hash for the entry.
for _, value := range []string{item.ID, item.URL, item.ContentText + item.ContentHTML + item.Summary} {
value = strings.TrimSpace(value)
+13 -6
View File
@@ -790,7 +790,9 @@ func TestParseItemTags(t *testing.T) {
"tags": [
" tag 1",
" ",
"tag 2"
"tag 2",
"tag 2",
"aaa"
]
}
]
@@ -801,14 +803,19 @@ func TestParseItemTags(t *testing.T) {
t.Fatal(err)
}
if len(feed.Entries[0].Tags) != 2 {
if len(feed.Entries) != 1 {
t.Errorf("Incorrect number of entries, got: %d", len(feed.Entries))
}
if len(feed.Entries[0].Tags) != 3 {
t.Errorf("Incorrect number of Tags, got: %d", len(feed.Entries[0].Tags))
}
expected := "tag 2"
result := feed.Entries[0].Tags[1]
if result != expected {
t.Errorf("Incorrect entry tag, got %q instead of %q", result, expected)
expected := []string{"aaa", "tag 1", "tag 2"}
for i, tag := range feed.Entries[0].Tags {
if tag != expected[i] {
t.Errorf("Incorrect entry tag, got %q instead of %q", tag, expected[i])
}
}
}
+35 -5
View File
@@ -4,9 +4,9 @@
package parser // import "miniflux.app/v2/internal/reader/parser"
import (
"bytes"
"encoding/xml"
"io"
"unicode"
rxml "miniflux.app/v2/internal/reader/xml"
)
@@ -22,10 +22,7 @@ const (
// DetectFeedFormat tries to guess the feed format from input data.
func DetectFeedFormat(r io.ReadSeeker) (string, string) {
data := make([]byte, 512)
r.Read(data)
if bytes.HasPrefix(bytes.TrimSpace(data), []byte("{")) {
if isJSON, err := detectJSONFormat(r); err == nil && isJSON {
return FormatJSON, ""
}
@@ -57,3 +54,36 @@ func DetectFeedFormat(r io.ReadSeeker) (string, string) {
return FormatUnknown, ""
}
// detectJSONFormat checks if the reader contains JSON by reading until it finds
// the first non-whitespace character or reaches EOF/error.
func detectJSONFormat(r io.ReadSeeker) (bool, error) {
const bufferSize = 32
buffer := make([]byte, bufferSize)
for {
n, err := r.Read(buffer)
if n == 0 {
if err == io.EOF {
return false, nil // No non-whitespace content found
}
return false, err
}
// Check each byte in the buffer
for i := range n {
ch := buffer[i]
// Skip whitespace characters (space, tab, newline, carriage return, etc.)
if unicode.IsSpace(rune(ch)) {
continue
}
// First non-whitespace character determines if it's JSON
return ch == '{', nil
}
// If we've read less than bufferSize, we've reached EOF
if n < bufferSize {
return false, nil
}
}
}
+53
View File
@@ -77,3 +77,56 @@ func TestDetectUnknown(t *testing.T) {
t.Errorf(`Wrong format detected: %q instead of %q`, format, FormatUnknown)
}
}
func TestDetectJSONWithLargeLeadingWhitespace(t *testing.T) {
leadingWhitespace := strings.Repeat(" ", 10000)
data := leadingWhitespace + `{
"version" : "https://jsonfeed.org/version/1",
"title" : "Example with lots of leading whitespace"
}`
format, _ := DetectFeedFormat(strings.NewReader(data))
if format != FormatJSON {
t.Errorf(`Wrong format detected: %q instead of %q`, format, FormatJSON)
}
}
func TestDetectJSONWithMixedWhitespace(t *testing.T) {
leadingWhitespace := strings.Repeat("\n\t ", 10000)
data := leadingWhitespace + `{
"version" : "https://jsonfeed.org/version/1",
"title" : "Example with mixed whitespace"
}`
format, _ := DetectFeedFormat(strings.NewReader(data))
if format != FormatJSON {
t.Errorf(`Wrong format detected: %q instead of %q`, format, FormatJSON)
}
}
func TestDetectOnlyWhitespace(t *testing.T) {
data := strings.Repeat(" \t\n\r", 10000)
format, _ := DetectFeedFormat(strings.NewReader(data))
if format != FormatUnknown {
t.Errorf(`Wrong format detected: %q instead of %q`, format, FormatUnknown)
}
}
func TestDetectJSONSmallerThanBuffer(t *testing.T) {
data := `{"version":"1"}` // This is only 15 bytes, well below the 32-byte buffer
format, _ := DetectFeedFormat(strings.NewReader(data))
if format != FormatJSON {
t.Errorf(`Wrong format detected: %q instead of %q`, format, FormatJSON)
}
}
func TestDetectJSONWithWhitespaceSmallerThanBuffer(t *testing.T) {
data := ` {"title":"test"} `
format, _ := DetectFeedFormat(strings.NewReader(data))
if format != FormatJSON {
t.Errorf(`Wrong format detected: %q instead of %q`, format, FormatJSON)
}
}
@@ -1,17 +0,0 @@
// SPDX-FileCopyrightText: Copyright The Miniflux Authors. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package processor // import "miniflux.app/v2/internal/reader/processor"
import (
"testing"
)
func TestMinifyEntryContent(t *testing.T) {
input := `<p> Some text with a <a href="http://example.org/"> link </a> </p>`
expected := `<p>Some text with a <a href="http://example.org/">link</a></p>`
result := minifyContent(input)
if expected != result {
t.Errorf(`Unexpected result, got %q`, result)
}
}
+1 -1
View File
@@ -43,7 +43,7 @@ func fetchWatchTime(websiteURL, query string, isoDate bool) (int, error) {
ret := 0
if isoDate {
parsedDuration, err := parseISO8601(duration)
parsedDuration, err := parseISO8601Duration(duration)
if err != nil {
return 0, fmt.Errorf("unable to parse iso duration %s: %v", duration, err)
}
+38 -32
View File
@@ -6,53 +6,56 @@ package processor // import "miniflux.app/v2/internal/reader/processor"
import (
"errors"
"fmt"
"regexp"
"strconv"
"strings"
"time"
"github.com/tdewolff/minify/v2"
"github.com/tdewolff/minify/v2/html"
)
// TODO: use something less horrible than a regex to parse ISO 8601 durations.
// parseISO8601Duration parses a subset of ISO8601 durations, mainly for youtube video.
func parseISO8601Duration(duration string) (time.Duration, error) {
after, ok := strings.CutPrefix(duration, "PT")
if !ok {
return 0, errors.New("the period doesn't start with PT")
}
var (
iso8601Regex = regexp.MustCompile(`^P((?P<year>\d+)Y)?((?P<month>\d+)M)?((?P<week>\d+)W)?((?P<day>\d+)D)?(T((?P<hour>\d+)H)?((?P<minute>\d+)M)?((?P<second>\d+)S)?)?$`)
)
func parseISO8601(from string) (time.Duration, error) {
var match []string
var d time.Duration
num := ""
if iso8601Regex.MatchString(from) {
match = iso8601Regex.FindStringSubmatch(from)
} else {
return 0, errors.New("processor: could not parse duration string")
}
for _, char := range after {
var val float64
var err error
for i, name := range iso8601Regex.SubexpNames() {
part := match[i]
if i == 0 || name == "" || part == "" {
continue
}
val, err := strconv.ParseInt(part, 10, 64)
if err != nil {
return 0, err
}
switch name {
case "hour":
switch char {
case 'Y', 'W', 'D':
return 0, fmt.Errorf("the '%c' specifier isn't supported", char)
case 'H':
if val, err = strconv.ParseFloat(num, 64); err != nil {
return 0, err
}
d += time.Duration(val) * time.Hour
case "minute":
num = ""
case 'M':
if val, err = strconv.ParseFloat(num, 64); err != nil {
return 0, err
}
d += time.Duration(val) * time.Minute
case "second":
num = ""
case 'S':
if val, err = strconv.ParseFloat(num, 64); err != nil {
return 0, err
}
d += time.Duration(val) * time.Second
num = ""
case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '.':
num += string(char)
continue
default:
return 0, fmt.Errorf("processor: unknown field %s", name)
return 0, errors.New("invalid character in the period")
}
}
return d, nil
}
@@ -61,8 +64,11 @@ func minifyContent(content string) string {
// Options required to avoid breaking the HTML content.
m.Add("text/html", &html.Minifier{
KeepEndTags: true,
KeepQuotes: true,
KeepEndTags: true,
KeepQuotes: true,
KeepComments: false,
KeepSpecialComments: false,
KeepDefaultAttrVals: false,
})
if minifiedHTML, err := m.String("text/html", content); err == nil {
+127
View File
@@ -0,0 +1,127 @@
// SPDX-FileCopyrightText: Copyright The Miniflux Authors. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package processor // import "miniflux.app/v2/internal/reader/processor"
import (
"testing"
"time"
)
func TestISO8601DurationParsing(t *testing.T) {
var scenarios = []struct {
duration string
expected time.Duration
}{
// Live streams and radio.
{"PT0M0S", 0},
// https://www.youtube.com/watch?v=HLrqNhgdiC0
{"PT6M20S", (6 * time.Minute) + (20 * time.Second)},
// https://www.youtube.com/watch?v=LZa5KKfqHtA
{"PT5M41S", (5 * time.Minute) + (41 * time.Second)},
// https://www.youtube.com/watch?v=yIxEEgEuhT4
{"PT51M52S", (51 * time.Minute) + (52 * time.Second)},
// https://www.youtube.com/watch?v=bpHf1XcoiFs
{"PT80M42S", (1 * time.Hour) + (20 * time.Minute) + (42 * time.Second)},
// Hours only
{"PT2H", 2 * time.Hour},
// Seconds only
{"PT30S", 30 * time.Second},
// Hours and minutes
{"PT1H30M", (1 * time.Hour) + (30 * time.Minute)},
// Hours and seconds
{"PT2H45S", (2 * time.Hour) + (45 * time.Second)},
// Empty duration
{"PT", 0},
}
for _, tc := range scenarios {
result, err := parseISO8601Duration(tc.duration)
if err != nil {
t.Errorf("Got an error when parsing %q: %v", tc.duration, err)
}
if tc.expected != result {
t.Errorf(`Unexpected result, got %v for duration %q`, result, tc.duration)
}
}
}
func TestISO8601DurationParsingErrors(t *testing.T) {
var errorScenarios = []struct {
duration string
expectedErr string
}{
// Missing PT prefix
{"6M20S", "the period doesn't start with PT"},
// Unsupported Year specifier
{"PT1Y", "the 'Y' specifier isn't supported"},
// Unsupported Week specifier
{"PT2W", "the 'W' specifier isn't supported"},
// Unsupported Day specifier
{"PT3D", "the 'D' specifier isn't supported"},
// Invalid number for hours (letter at start of number)
{"PTaH", "invalid character in the period"},
// Invalid number for minutes (letter at start of number)
{"PTbM", "invalid character in the period"},
// Invalid number for seconds (letter at start of number)
{"PTcS", "invalid character in the period"},
// Invalid character in the middle of a number
{"PT1a2H", "invalid character in the period"},
{"PT3b4M", "invalid character in the period"},
{"PT5c6S", "invalid character in the period"},
// Test cases for actual ParseFloat errors (empty number before specifier)
{"PTH", "strconv.ParseFloat: parsing \"\": invalid syntax"},
{"PTM", "strconv.ParseFloat: parsing \"\": invalid syntax"},
{"PTS", "strconv.ParseFloat: parsing \"\": invalid syntax"},
// Invalid character
{"PT1X", "invalid character in the period"},
// Invalid character mixed
{"PT1H@M", "invalid character in the period"},
}
for _, tc := range errorScenarios {
_, err := parseISO8601Duration(tc.duration)
if err == nil {
t.Errorf("Expected an error when parsing %q, but got none", tc.duration)
} else if err.Error() != tc.expectedErr {
t.Errorf("Expected error %q when parsing %q, but got %q", tc.expectedErr, tc.duration, err.Error())
}
}
}
func TestMinifyEntryContentWithWhitespace(t *testing.T) {
input := `<p> Some text with a <a href="http://example.org/"> link </a> </p>`
expected := `<p>Some text with a <a href="http://example.org/">link</a></p>`
result := minifyContent(input)
if expected != result {
t.Errorf(`Unexpected result, got %q`, result)
}
}
func TestMinifyContentWithDefaultAttributes(t *testing.T) {
input := `<script type="application/javascript">console.log("Hello, World!");</script>`
expected := `<script>console.log("Hello, World!");</script>`
result := minifyContent(input)
if expected != result {
t.Errorf(`Unexpected result, got %q`, result)
}
}
func TestMinifyContentWithComments(t *testing.T) {
input := `<p>Some text<!-- This is a comment --> with a <a href="http://example.org/">link</a>.</p>`
expected := `<p>Some text with a <a href="http://example.org/">link</a>.</p>`
result := minifyContent(input)
if expected != result {
t.Errorf(`Unexpected result, got %q`, result)
}
}
func TestMinifyContentWithSpecialComments(t *testing.T) {
input := `<p>Some text <!--[if IE 6]><p>IE6</p><![endif]--> with a <a href="http://example.org/">link</a>.</p>`
expected := `<p>Some text with a <a href="http://example.org/">link</a>.</p>`
result := minifyContent(input)
if expected != result {
t.Errorf(`Unexpected result, got %q`, result)
}
}
+1 -1
View File
@@ -118,7 +118,7 @@ func fetchYouTubeWatchTimeFromApiInBulk(videoIDs []string) (map[string]time.Dura
watchTimeMap := make(map[string]time.Duration, len(videos.Items))
for _, video := range videos.Items {
duration, err := parseISO8601(video.ContentDetails.Duration)
duration, err := parseISO8601Duration(video.ContentDetails.Duration)
if err != nil {
slog.Warn("Unable to parse ISO8601 duration", slog.Any("error", err))
continue
-30
View File
@@ -5,38 +5,8 @@ package processor // import "miniflux.app/v2/internal/reader/processor"
import (
"testing"
"time"
)
func TestParseISO8601(t *testing.T) {
var scenarios = []struct {
duration string
expected time.Duration
}{
// Live streams and radio.
{"PT0M0S", 0},
// https://www.youtube.com/watch?v=HLrqNhgdiC0
{"PT6M20S", (6 * time.Minute) + (20 * time.Second)},
// https://www.youtube.com/watch?v=LZa5KKfqHtA
{"PT5M41S", (5 * time.Minute) + (41 * time.Second)},
// https://www.youtube.com/watch?v=yIxEEgEuhT4
{"PT51M52S", (51 * time.Minute) + (52 * time.Second)},
// https://www.youtube.com/watch?v=bpHf1XcoiFs
{"PT80M42S", (1 * time.Hour) + (20 * time.Minute) + (42 * time.Second)},
}
for _, tc := range scenarios {
result, err := parseISO8601(tc.duration)
if err != nil {
t.Errorf("Got an error when parsing %q: %v", tc.duration, err)
}
if tc.expected != result {
t.Errorf(`Unexpected result, got %v for duration %q`, result, tc.duration)
}
}
}
func TestGetYouTubeVideoIDFromURL(t *testing.T) {
scenarios := []struct {
url string
+162 -108
View File
@@ -7,7 +7,6 @@ import (
"fmt"
"io"
"log/slog"
"regexp"
"strings"
"miniflux.app/v2/internal/urllib"
@@ -16,18 +15,15 @@ import (
"golang.org/x/net/html"
)
const (
defaultTagsToScore = "section,h2,h3,h4,h5,h6,p,td,pre,div"
)
const defaultTagsToScore = "section,h2,h3,h4,h5,h6,p,td,pre,div"
var (
divToPElementsRegexp = regexp.MustCompile(`(?i)<(?:a|blockquote|dl|div|img|ol|p|pre|table|ul)[ />]`)
strongCandidatesToRemove = [...]string{"popupbody", "-ad", "g-plus"}
maybeCandidateToRemove = [...]string{"and", "article", "body", "column", "main", "shadow", "content"}
unlikelyCandidateToRemove = [...]string{"banner", "breadcrumbs", "combx", "comment", "community", "cover-wrap", "disqus", "extra", "foot", "header", "legends", "menu", "modal", "related", "remark", "replies", "rss", "shoutbox", "sidebar", "skyscraper", "social", "sponsor", "supplemental", "ad-break", "agegate", "pagination", "pager", "popup", "yom-remote"}
okMaybeItsACandidateRegexp = regexp.MustCompile(`and|article|body|column|main|shadow`)
unlikelyCandidatesRegexp = regexp.MustCompile(`banner|breadcrumbs|combx|comment|community|cover-wrap|disqus|extra|foot|header|legends|menu|modal|related|remark|replies|rss|shoutbox|sidebar|skyscraper|social|sponsor|supplemental|ad-break|agegate|pagination|pager|popup|yom-remote`)
negativeRegexp = regexp.MustCompile(`hid|banner|combx|comment|com-|contact|foot|masthead|media|meta|modal|outbrain|promo|related|scroll|share|shoutbox|sidebar|skyscraper|sponsor|shopping|tags|tool|widget|byline|author|dateline|writtenby`)
positiveRegexp = regexp.MustCompile(`article|body|content|entry|hentry|h-entry|main|page|pagination|post|text|blog|story`)
positiveKeywords = [...]string{"article", "blog", "body", "content", "entry", "h-entry", "hentry", "main", "page", "pagination", "post", "story", "text"}
negativeKeywords = [...]string{"author", "banner", "byline", "com-", "combx", "comment", "contact", "dateline", "foot", "hid", "masthead", "media", "meta", "modal", "outbrain", "promo", "related", "scroll", "share", "shopping", "shoutbox", "sidebar", "skyscraper", "sponsor", "tags", "tool", "widget", "writtenby"}
)
type candidate struct {
@@ -36,23 +32,31 @@ type candidate struct {
}
func (c *candidate) Node() *html.Node {
if c.selection.Length() == 0 {
return nil
}
return c.selection.Get(0)
}
func (c *candidate) String() string {
node := c.Node()
if node == nil {
return fmt.Sprintf("empty => %f", c.score)
}
id, _ := c.selection.Attr("id")
class, _ := c.selection.Attr("class")
switch {
case id != "" && class != "":
return fmt.Sprintf("%s#%s.%s => %f", c.Node().DataAtom, id, class, c.score)
return fmt.Sprintf("%s#%s.%s => %f", node.DataAtom, id, class, c.score)
case id != "":
return fmt.Sprintf("%s#%s => %f", c.Node().DataAtom, id, c.score)
return fmt.Sprintf("%s#%s => %f", node.DataAtom, id, c.score)
case class != "":
return fmt.Sprintf("%s.%s => %f", c.Node().DataAtom, class, c.score)
return fmt.Sprintf("%s.%s => %f", node.DataAtom, class, c.score)
}
return fmt.Sprintf("%s => %f", c.Node().DataAtom, c.score)
return fmt.Sprintf("%s => %f", node.DataAtom, c.score)
}
type candidateList map[*html.Node]*candidate
@@ -82,61 +86,93 @@ func ExtractContent(page io.Reader) (baseURL string, extractedContent string, er
document.Find("script,style").Remove()
transformMisusedDivsIntoParagraphs(document)
removeUnlikelyCandidates(document)
transformMisusedDivsIntoParagraphs(document)
candidates := getCandidates(document)
topCandidate := getTopCandidate(document, candidates)
slog.Debug("Readability parsing",
slog.String("base_url", baseURL),
slog.Any("candidates", candidates),
slog.Any("topCandidate", topCandidate),
slog.String("candidates", candidates.String()),
slog.String("topCandidate", topCandidate.String()),
)
extractedContent = getArticle(topCandidate, candidates)
return baseURL, extractedContent, nil
}
func getSelectionLength(s *goquery.Selection) int {
return sumMapOnSelection(s, func(s string) int { return len(s) })
}
func getSelectionCommaCount(s *goquery.Selection) int {
return sumMapOnSelection(s, func(s string) int { return strings.Count(s, ",") })
}
// sumMapOnSelection maps `f` on the selection, and return the sum of the result.
// This construct is used instead of goquery.Selection's .Text() method,
// to avoid materializing the text to simply map/sum on it, saving a significant
// amount of memory of large selections, and reducing the pressure on the garbage-collector.
func sumMapOnSelection(s *goquery.Selection, f func(str string) int) int {
var recursiveFunction func(*html.Node) int
recursiveFunction = func(n *html.Node) int {
total := 0
if n.Type == html.TextNode {
total += f(n.Data)
}
if n.FirstChild != nil {
for c := n.FirstChild; c != nil; c = c.NextSibling {
total += recursiveFunction(c)
}
}
return total
}
sum := 0
for _, n := range s.Nodes {
sum += recursiveFunction(n)
}
return sum
}
// Now that we have the top candidate, look through its siblings for content that might also be related.
// Things like preambles, content split by ads that we removed, etc.
func getArticle(topCandidate *candidate, candidates candidateList) string {
var output strings.Builder
output.WriteString("<div>")
siblingScoreThreshold := max(10, topCandidate.score*.2)
siblingScoreThreshold := max(10, topCandidate.score/5)
topCandidate.selection.Siblings().Union(topCandidate.selection).Each(func(i int, s *goquery.Selection) {
append := false
tag := "div"
node := s.Get(0)
if node == topCandidate.Node() {
topNode := topCandidate.Node()
if topNode != nil && node == topNode {
append = true
} else if c, ok := candidates[node]; ok && c.score >= siblingScoreThreshold {
append = true
}
if s.Is("p") {
} else if s.Is("p") {
tag = node.Data
linkDensity := getLinkDensity(s)
content := s.Text()
contentLength := len(content)
contentLength := getSelectionLength(s)
if contentLength >= 80 {
if linkDensity < .25 {
append = true
}
} else {
if linkDensity == 0 && containsSentence(content) {
append = true
if linkDensity == 0 {
// It's a small selection, so .Text doesn't impact performances too much.
if containsSentence(s.Text()) {
append = true
}
}
}
}
if append {
tag := "div"
if s.Is("p") {
tag = node.Data
}
html, _ := s.Html()
output.WriteString("<" + tag + ">" + html + "</" + tag + ">")
}
@@ -145,38 +181,53 @@ func getArticle(topCandidate *candidate, candidates candidateList) string {
output.WriteString("</div>")
return output.String()
}
func shouldRemoveCandidate(str string) bool {
str = strings.ToLower(str)
func removeUnlikelyCandidates(document *goquery.Document) {
var shouldRemove = func(str string) bool {
str = strings.ToLower(str)
if strings.Contains(str, "popupbody") || strings.Contains(str, "-ad") || strings.Contains(str, "g-plus") {
return true
} else if unlikelyCandidatesRegexp.MatchString(str) && !okMaybeItsACandidateRegexp.MatchString(str) {
// Those candidates have no false-positives, no need to check against `maybeCandidate`
for _, strongCandidateToRemove := range strongCandidatesToRemove {
if strings.Contains(str, strongCandidateToRemove) {
return true
}
return false
}
document.Find("*").Each(func(i int, s *goquery.Selection) {
if s.Length() == 0 || s.Get(0).Data == "html" || s.Get(0).Data == "body" {
return
for _, unlikelyCandidateToRemove := range unlikelyCandidateToRemove {
if strings.Contains(str, unlikelyCandidateToRemove) {
// Do we have a false positive?
for _, maybeCandidateToRemove := range maybeCandidateToRemove {
if strings.Contains(str, maybeCandidateToRemove) {
return false
}
}
// Nope, it's a true positive!
return true
}
}
return false
}
func removeUnlikelyCandidates(document *goquery.Document) {
// Only select tags with either a class or an id attribute,
// and never the html nor body tags, as we don't want to ever remove them.
selector := "[class]:not(body,html)" + "," + "[id]:not(body,html)"
for _, s := range document.Find(selector).EachIter() {
if s.Length() == 0 {
continue
}
// Don't remove elements within code blocks (pre or code tags)
if s.Closest("pre, code").Length() > 0 {
return
if s.Closest("pre,code").Length() > 0 {
continue
}
if class, ok := s.Attr("class"); ok {
if shouldRemove(class) {
s.Remove()
}
} else if id, ok := s.Attr("id"); ok {
if shouldRemove(id) {
s.Remove()
}
if class, ok := s.Attr("class"); ok && shouldRemoveCandidate(class) {
s.Remove()
} else if id, ok := s.Attr("id"); ok && shouldRemoveCandidate(id) {
s.Remove()
}
})
}
}
func getTopCandidate(document *goquery.Document, candidates candidateList) *candidate {
@@ -200,49 +251,41 @@ func getTopCandidate(document *goquery.Document, candidates candidateList) *cand
// Loop through all paragraphs, and assign a score to them based on how content-y they look.
// Then add their score to their parent node.
// A score is determined by things like number of commas, class names, etc.
// Maybe eventually link density.
func getCandidates(document *goquery.Document) candidateList {
candidates := make(candidateList)
document.Find(defaultTagsToScore).Each(func(i int, s *goquery.Selection) {
text := s.Text()
textLen := getSelectionLength(s)
// If this paragraph is less than 25 characters, don't even count it.
if len(text) < 25 {
if textLen < 25 {
return
}
// Add a point for the paragraph itself as a base.
contentScore := 1
// Add points for any commas within this paragraph.
contentScore += getSelectionCommaCount(s) + 1
// For every 100 characters in this paragraph, add another point. Up to 3 points.
contentScore += min(textLen/100, 3)
parent := s.Parent()
parentNode := parent.Get(0)
grandParent := parent.Parent()
var grandParentNode *html.Node
if grandParent.Length() > 0 {
grandParentNode = grandParent.Get(0)
}
if _, found := candidates[parentNode]; !found {
candidates[parentNode] = scoreNode(parent)
}
candidates[parentNode].score += float32(contentScore)
if grandParentNode != nil {
// The score of the current node influences its grandparent's one as well, but scaled to 50%.
grandParent := parent.Parent()
if grandParent.Length() > 0 {
grandParentNode := grandParent.Get(0)
if _, found := candidates[grandParentNode]; !found {
candidates[grandParentNode] = scoreNode(grandParent)
}
}
// Add a point for the paragraph itself as a base.
contentScore := float32(1.0)
// Add points for any commas within this paragraph.
contentScore += float32(strings.Count(text, ",") + 1)
// For every 100 characters in this paragraph, add another point. Up to 3 points.
contentScore += float32(min(len(text)/100.0, 3))
candidates[parentNode].score += contentScore
if grandParentNode != nil {
candidates[grandParentNode].score += contentScore / 2.0
candidates[grandParentNode].score += float32(contentScore) / 2.0
}
})
@@ -259,7 +302,12 @@ func getCandidates(document *goquery.Document) candidateList {
func scoreNode(s *goquery.Selection) *candidate {
c := &candidate{selection: s, score: 0}
switch s.Get(0).DataAtom.String() {
// Check if selection is empty to avoid panic
if s.Length() == 0 {
return c
}
switch s.Get(0).Data {
case "div":
c.score += 5
case "pre", "td", "blockquote", "img":
@@ -270,56 +318,62 @@ func scoreNode(s *goquery.Selection) *candidate {
c.score -= 5
}
c.score += getClassWeight(s)
if class, ok := s.Attr("class"); ok {
c.score += getWeight(class)
}
if id, ok := s.Attr("id"); ok {
c.score += getWeight(id)
}
return c
}
// Get the density of links as a percentage of the content
// This is the amount of text that is inside a link divided by the total text in the node.
func getLinkDensity(s *goquery.Selection) float32 {
textLength := len(s.Text())
if textLength == 0 {
sum := getSelectionLength(s)
if sum == 0 {
return 0
}
linkLength := len(s.Find("a").Text())
linkLength := getSelectionLength(s.Find("a"))
return float32(linkLength) / float32(textLength)
return float32(linkLength) / float32(sum)
}
// Get an elements class/id weight. Uses regular expressions to tell if this
// element looks good or bad.
func getClassWeight(s *goquery.Selection) float32 {
weight := 0
if class, ok := s.Attr("class"); ok {
class = strings.ToLower(class)
if negativeRegexp.MatchString(class) {
weight -= 25
} else if positiveRegexp.MatchString(class) {
weight += 25
func getWeight(s string) float32 {
s = strings.ToLower(s)
for _, keyword := range negativeKeywords {
if strings.Contains(s, keyword) {
return -25
}
}
if id, ok := s.Attr("id"); ok {
id = strings.ToLower(id)
if negativeRegexp.MatchString(id) {
weight -= 25
} else if positiveRegexp.MatchString(id) {
weight += 25
for _, keyword := range positiveKeywords {
if strings.Contains(s, keyword) {
return +25
}
}
return float32(weight)
return 0
}
func transformMisusedDivsIntoParagraphs(document *goquery.Document) {
document.Find("div").Each(func(i int, s *goquery.Selection) {
html, _ := s.Html()
if !divToPElementsRegexp.MatchString(html) {
node := s.Get(0)
node.Data = "p"
nodes := s.Children().Nodes
if len(nodes) == 0 {
s.Nodes[0].Data = "p"
return
}
for _, node := range nodes {
switch node.Data {
case "a", "blockquote", "div", "dl",
"img", "ol", "p", "pre",
"table", "ul":
return
default:
s.Nodes[0].Data = "p"
}
}
})
}
File diff suppressed because it is too large Load Diff
+28 -26
View File
@@ -10,38 +10,40 @@ import (
"miniflux.app/v2/internal/model"
)
var customReplaceRuleRegex = regexp.MustCompile(`rewrite\("([^"]+)"\|"([^"]+)"\)`)
var customReplaceRuleRegex = regexp.MustCompile(`^rewrite\("([^"]+)"\|"([^"]+)"\)$`)
func RewriteEntryURL(feed *model.Feed, entry *model.Entry) string {
var rewrittenURL = entry.URL
if feed.UrlRewriteRules != "" {
parts := customReplaceRuleRegex.FindStringSubmatch(feed.UrlRewriteRules)
if feed.UrlRewriteRules == "" {
return entry.URL
}
if len(parts) >= 3 {
re, err := regexp.Compile(parts[1])
if err != nil {
slog.Error("Failed on regexp compilation",
slog.String("url_rewrite_rules", feed.UrlRewriteRules),
slog.Any("error", err),
)
return rewrittenURL
}
rewrittenURL = re.ReplaceAllString(entry.URL, parts[2])
slog.Debug("Rewriting entry URL",
slog.String("original_entry_url", entry.URL),
slog.String("rewritten_entry_url", rewrittenURL),
slog.Int64("feed_id", feed.ID),
slog.String("feed_url", feed.FeedURL),
)
} else {
slog.Debug("Cannot find search and replace terms for replace rule",
slog.String("original_entry_url", entry.URL),
slog.String("rewritten_entry_url", rewrittenURL),
slog.Int64("feed_id", feed.ID),
slog.String("feed_url", feed.FeedURL),
var rewrittenURL = entry.URL
parts := customReplaceRuleRegex.FindStringSubmatch(feed.UrlRewriteRules)
if len(parts) == 3 {
re, err := regexp.Compile(parts[1])
if err != nil {
slog.Error("Failed on regexp compilation",
slog.String("url_rewrite_rules", feed.UrlRewriteRules),
slog.Any("error", err),
)
return rewrittenURL
}
rewrittenURL = re.ReplaceAllString(entry.URL, parts[2])
slog.Debug("Rewriting entry URL",
slog.String("original_entry_url", entry.URL),
slog.String("rewritten_entry_url", rewrittenURL),
slog.Int64("feed_id", feed.ID),
slog.String("feed_url", feed.FeedURL),
)
} else {
slog.Debug("Cannot find search and replace terms for replace rule",
slog.String("original_entry_url", entry.URL),
slog.String("rewritten_entry_url", rewrittenURL),
slog.Int64("feed_id", feed.ID),
slog.String("feed_url", feed.FeedURL),
slog.String("url_rewrite_rules", feed.UrlRewriteRules),
)
}
return rewrittenURL
+57 -24
View File
@@ -7,6 +7,7 @@ import (
"html"
"log/slog"
"path"
"slices"
"strconv"
"strings"
"time"
@@ -78,7 +79,13 @@ func (r *RSSAdapter) BuildFeed(baseURL string) *model.Feed {
// Populate the entry URL.
entryURL := findEntryURL(&item)
if entryURL == "" {
entry.URL = feed.SiteURL
// Fallback to the first enclosure URL if it exists.
if len(entry.Enclosures) > 0 && entry.Enclosures[0].URL != "" {
entry.URL = entry.Enclosures[0].URL
} else {
// Fallback to the feed URL if no entry URL is found.
entry.URL = feed.SiteURL
}
} else {
if absoluteEntryURL, err := urllib.AbsoluteURL(feed.SiteURL, entryURL); err == nil {
entry.URL = absoluteEntryURL
@@ -124,31 +131,13 @@ func (r *RSSAdapter) BuildFeed(baseURL string) *model.Feed {
}
// Populate entry categories.
for _, tag := range item.Categories {
if tag != "" {
entry.Tags = append(entry.Tags, tag)
}
}
for _, tag := range item.MediaCategories.Labels() {
if tag != "" {
entry.Tags = append(entry.Tags, tag)
}
}
entry.Tags = findEntryTags(&item)
if len(entry.Tags) == 0 {
for _, tag := range r.rss.Channel.Categories {
if tag != "" {
entry.Tags = append(entry.Tags, tag)
}
}
for _, tag := range r.rss.Channel.GetItunesCategories() {
if tag != "" {
entry.Tags = append(entry.Tags, tag)
}
}
if r.rss.Channel.GooglePlayCategory.Text != "" {
entry.Tags = append(entry.Tags, r.rss.Channel.GooglePlayCategory.Text)
}
entry.Tags = findFeedTags(&r.rss.Channel)
}
// Sort and deduplicate tags.
slices.Sort(entry.Tags)
entry.Tags = slices.Compact(entry.Tags)
feed.Entries = append(feed.Entries, entry)
}
@@ -176,6 +165,30 @@ func findFeedAuthor(rssChannel *RSSChannel) string {
return strings.TrimSpace(sanitizer.StripTags(author))
}
func findFeedTags(rssChannel *RSSChannel) []string {
tags := make([]string, 0)
for _, tag := range rssChannel.Categories {
tag = strings.TrimSpace(tag)
if tag != "" {
tags = append(tags, tag)
}
}
for _, tag := range rssChannel.GetItunesCategories() {
tag = strings.TrimSpace(tag)
if tag != "" {
tags = append(tags, tag)
}
}
if tag := strings.TrimSpace(rssChannel.GooglePlayCategory.Text); tag != "" {
tags = append(tags, tag)
}
return tags
}
func findEntryTitle(rssItem *RSSItem) string {
title := rssItem.Title.Content
@@ -270,6 +283,26 @@ func findEntryAuthor(rssItem *RSSItem) string {
return strings.TrimSpace(sanitizer.StripTags(author))
}
func findEntryTags(rssItem *RSSItem) []string {
tags := make([]string, 0)
for _, tag := range rssItem.Categories {
tag = strings.TrimSpace(tag)
if tag != "" {
tags = append(tags, tag)
}
}
for _, tag := range rssItem.MediaCategories.Labels() {
tag = strings.TrimSpace(tag)
if tag != "" {
tags = append(tags, tag)
}
}
return tags
}
func findEntryEnclosures(rssItem *RSSItem, siteURL string) model.EnclosureList {
enclosures := make(model.EnclosureList, 0)
duplicates := make(map[string]bool)
+36 -5
View File
@@ -632,6 +632,34 @@ func TestParseEntryWithMultipleAtomLinks(t *testing.T) {
}
}
func TestParseEntryWithoutLinkAndWithEnclosureURLs(t *testing.T) {
data := `<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0">
<channel>
<link>https://example.org/feed</link>
<item>
<guid isPermaLink="false">guid</guid>
<enclosure url=" " length="155844084" type="audio/mpeg" />
<enclosure url="https://audio-file" length="155844084" type="audio/mpeg" />
<enclosure url="https://another-audio-file" length="155844084" type="audio/mpeg" />
</item>
</channel>
</rss>`
feed, err := Parse("https://example.org/", bytes.NewReader([]byte(data)))
if err != nil {
t.Fatal(err)
}
if len(feed.Entries) != 1 {
t.Fatalf("Expected 1 entry, got: %d", len(feed.Entries))
}
if feed.Entries[0].URL != "https://audio-file" {
t.Errorf("Incorrect entry link, got: %q", feed.Entries[0].URL)
}
}
func TestParseFeedURLWithAtomLink(t *testing.T) {
data := `<?xml version="1.0" encoding="utf-8"?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
@@ -1971,6 +1999,9 @@ func TestParseEntryWithCategories(t *testing.T) {
<link>https://example.org/item</link>
<category>Category 1</category>
<category><![CDATA[Category 2]]></category>
<category>Category 2</category>
<category>Category 0</category>
<category> </category>
</item>
</channel>
</rss>`
@@ -1980,11 +2011,11 @@ func TestParseEntryWithCategories(t *testing.T) {
t.Fatal(err)
}
if len(feed.Entries[0].Tags) != 2 {
if len(feed.Entries[0].Tags) != 3 {
t.Fatalf("Incorrect number of tags, got: %d", len(feed.Entries[0].Tags))
}
expected := []string{"Category 1", "Category 2"}
expected := []string{"Category 0", "Category 1", "Category 2"}
result := feed.Entries[0].Tags
for i, tag := range result {
@@ -2022,7 +2053,7 @@ func TestParseFeedWithItunesCategories(t *testing.T) {
t.Errorf("Incorrect number of tags, got: %d", len(feed.Entries[0].Tags))
}
expected := []string{"Society & Culture", "Documentary", "Health", "Mental Health"}
expected := []string{"Documentary", "Health", "Mental Health", "Society & Culture"}
result := feed.Entries[0].Tags
for i, tag := range result {
@@ -2091,12 +2122,12 @@ func TestParseEntryWithMediaCategories(t *testing.T) {
t.Errorf("Incorrect number of tags, got: %d", len(feed.Entries[0].Tags))
}
expected := []string{"Visual Art", "Ace Ventura - Pet Detective"}
expected := []string{"Ace Ventura - Pet Detective", "Visual Art"}
result := feed.Entries[0].Tags
for i, tag := range result {
if tag != expected[i] {
t.Errorf("Incorrect tag, got: %q", tag)
t.Errorf("Incorrect entry tag, got %q instead of %q", tag, expected[i])
}
}
}
+49 -24
View File
@@ -204,10 +204,15 @@ func SanitizeHTMLWithDefaultOptions(baseURL, rawHTML string) string {
}
func SanitizeHTML(baseURL, rawHTML string, sanitizerOptions *SanitizerOptions) string {
var buffer strings.Builder
var tagStack []string
var parentTag string
var blockedStack []string
var buffer strings.Builder
// Educated guess about how big the sanitized HTML will be,
// to reduce the amount of buffer re-allocations in this function.
estimatedRatio := len(rawHTML) * 3 / 4
buffer.Grow(estimatedRatio)
// Errors are a non-issue, so they're handled later in the function.
parsedBaseUrl, _ := url.Parse(baseURL)
@@ -259,7 +264,7 @@ func SanitizeHTML(baseURL, rawHTML string, sanitizerOptions *SanitizerOptions) s
}
if len(blockedStack) == 0 && isValidTag(tagName) {
attrNames, htmlAttributes := sanitizeAttributes(parsedBaseUrl, baseURL, tagName, token.Attr, sanitizerOptions)
attrNames, htmlAttributes := sanitizeAttributes(parsedBaseUrl, tagName, token.Attr, sanitizerOptions)
if hasRequiredAttributes(tagName, attrNames) {
if len(attrNames) > 0 {
// Rewrite the start tag with allowed attributes.
@@ -287,7 +292,7 @@ func SanitizeHTML(baseURL, rawHTML string, sanitizerOptions *SanitizerOptions) s
continue
}
if len(blockedStack) == 0 && isValidTag(tagName) {
attrNames, htmlAttributes := sanitizeAttributes(parsedBaseUrl, baseURL, tagName, token.Attr, sanitizerOptions)
attrNames, htmlAttributes := sanitizeAttributes(parsedBaseUrl, tagName, token.Attr, sanitizerOptions)
if hasRequiredAttributes(tagName, attrNames) {
if len(attrNames) > 0 {
buffer.WriteString("<" + tagName + " " + htmlAttributes + "/>")
@@ -300,29 +305,26 @@ func SanitizeHTML(baseURL, rawHTML string, sanitizerOptions *SanitizerOptions) s
}
}
func sanitizeAttributes(parsedBaseUrl *url.URL, baseURL, tagName string, attributes []html.Attribute, sanitizerOptions *SanitizerOptions) ([]string, string) {
func sanitizeAttributes(parsedBaseUrl *url.URL, tagName string, attributes []html.Attribute, sanitizerOptions *SanitizerOptions) ([]string, string) {
var htmlAttrs, attrNames []string
var err error
var isImageLargerThanLayout bool
var isAnchorLink bool
if tagName == "img" {
imgWidth := getIntegerAttributeValue("width", attributes)
isImageLargerThanLayout = imgWidth > 750
}
for _, attribute := range attributes {
value := attribute.Val
if !isValidAttribute(tagName, attribute.Key) {
continue
}
if tagName == "math" && attribute.Key == "xmlns" && value != "http://www.w3.org/1998/Math/MathML" {
value = "http://www.w3.org/1998/Math/MathML"
}
value := attribute.Val
if tagName == "img" {
switch tagName {
case "math":
if attribute.Key == "xmlns" {
if value != "http://www.w3.org/1998/Math/MathML" {
value = "http://www.w3.org/1998/Math/MathML"
}
}
case "img":
switch attribute.Key {
case "fetchpriority":
if !isValidFetchPriorityValue(value) {
@@ -333,14 +335,21 @@ func sanitizeAttributes(parsedBaseUrl *url.URL, baseURL, tagName string, attribu
continue
}
case "width", "height":
if isImageLargerThanLayout || !isPositiveInteger(value) {
if !isPositiveInteger(value) {
continue
}
}
}
if (tagName == "img" || tagName == "source") && attribute.Key == "srcset" {
value = sanitizeSrcsetAttr(baseURL, value)
// Discard width and height attributes when width is larger than Miniflux layout (750px)
if imgWidth := getIntegerAttributeValue("width", attributes); imgWidth > 750 {
continue
}
case "srcset":
value = sanitizeSrcsetAttr(parsedBaseUrl, value)
}
case "source":
if attribute.Key == "srcset" {
value = sanitizeSrcsetAttr(parsedBaseUrl, value)
}
}
if isExternalResourceAttribute(attribute.Key) {
@@ -356,7 +365,7 @@ func sanitizeAttributes(parsedBaseUrl *url.URL, baseURL, tagName string, attribu
value = attribute.Val
isAnchorLink = true
default:
value, err = urllib.AbsoluteURL(baseURL, value)
value, err = absoluteURLParsedBase(parsedBaseUrl, value)
if err != nil {
continue
}
@@ -537,11 +546,11 @@ func isBlockedTag(tagName string) bool {
return false
}
func sanitizeSrcsetAttr(baseURL, value string) string {
func sanitizeSrcsetAttr(parsedBaseURL *url.URL, value string) string {
imageCandidates := ParseSrcSetAttribute(value)
for _, imageCandidate := range imageCandidates {
if absoluteURL, err := urllib.AbsoluteURL(baseURL, imageCandidate.ImageURL); err == nil {
if absoluteURL, err := absoluteURLParsedBase(parsedBaseURL, imageCandidate.ImageURL); err == nil {
imageCandidate.ImageURL = absoluteURL
}
}
@@ -593,3 +602,19 @@ func isValidDecodingValue(value string) bool {
}
return false
}
// absoluteURLParsedBase is used instead of urllib.AbsoluteURL to avoid parsing baseURL over and over.
func absoluteURLParsedBase(parsedBaseURL *url.URL, input string) (string, error) {
absURL, u, err := urllib.GetAbsoluteURL(input)
if err != nil {
return "", err
}
if absURL != "" {
return absURL, nil
}
if parsedBaseURL == nil {
return "", nil
}
return parsedBaseURL.ResolveReference(u).String(), nil
}
+42 -2
View File
@@ -73,7 +73,7 @@ func TestImgWithWidthAndHeightAttribute(t *testing.T) {
}
}
func TestImgWithWidthAndHeightAttributeLargerThanMinifluxLayout(t *testing.T) {
func TestImgWithWidthAttributeLargerThanMinifluxLayout(t *testing.T) {
input := `<img src="https://example.org/image.png" width="1200" height="675">`
expected := `<img src="https://example.org/image.png" loading="lazy">`
output := SanitizeHTMLWithDefaultOptions("http://example.org/", input)
@@ -93,7 +93,17 @@ func TestImgWithIncorrectWidthAndHeightAttribute(t *testing.T) {
}
}
func TestImgWithEmptywidthAndHeightAttribute(t *testing.T) {
func TestImgWithIncorrectWidthAttribute(t *testing.T) {
input := `<img src="https://example.org/image.png" width="10px" height="20">`
expected := `<img src="https://example.org/image.png" height="20" loading="lazy">`
output := SanitizeHTMLWithDefaultOptions("http://example.org/", input)
if output != expected {
t.Errorf(`Wrong output: %s`, output)
}
}
func TestImgWithEmptyWidthAndHeightAttribute(t *testing.T) {
input := `<img src="https://example.org/image.png" width="" height="">`
expected := `<img src="https://example.org/image.png" loading="lazy">`
output := SanitizeHTMLWithDefaultOptions("http://example.org/", input)
@@ -103,6 +113,36 @@ func TestImgWithEmptywidthAndHeightAttribute(t *testing.T) {
}
}
func TestImgWithIncorrectHeightAttribute(t *testing.T) {
input := `<img src="https://example.org/image.png" width="10" height="20px">`
expected := `<img src="https://example.org/image.png" width="10" loading="lazy">`
output := SanitizeHTMLWithDefaultOptions("http://example.org/", input)
if output != expected {
t.Errorf(`Wrong output: %s`, output)
}
}
func TestImgWithNegativeWidthAttribute(t *testing.T) {
input := `<img src="https://example.org/image.png" width="-10" height="20">`
expected := `<img src="https://example.org/image.png" height="20" loading="lazy">`
output := SanitizeHTMLWithDefaultOptions("http://example.org/", input)
if output != expected {
t.Errorf(`Wrong output: %s`, output)
}
}
func TestImgWithNegativeHeightAttribute(t *testing.T) {
input := `<img src="https://example.org/image.png" width="10" height="-20">`
expected := `<img src="https://example.org/image.png" width="10" loading="lazy">`
output := SanitizeHTMLWithDefaultOptions("http://example.org/", input)
if output != expected {
t.Errorf(`Wrong output: %s`, output)
}
}
func TestImgWithTextDataURL(t *testing.T) {
input := `<img src="data:text/plain;base64,SGVsbG8sIFdvcmxkIQ==" alt="Example">`
expected := ``
+3 -9
View File
@@ -5,11 +5,9 @@ package subscription // import "miniflux.app/v2/internal/reader/subscription"
import (
"bytes"
"fmt"
"io"
"log/slog"
"net/url"
"regexp"
"strings"
"miniflux.app/v2/internal/config"
@@ -24,10 +22,6 @@ import (
"github.com/PuerkitoBio/goquery"
)
var (
youtubeChannelRegex = regexp.MustCompile(`channel/(.*)$`)
)
type SubscriptionFinder struct {
requestBuilder *fetcher.RequestBuilder
feedDownloaded bool
@@ -300,8 +294,8 @@ func (f *SubscriptionFinder) FindSubscriptionsFromYouTubeChannelPage(websiteURL
return nil, nil
}
if matches := youtubeChannelRegex.FindStringSubmatch(decodedUrl.Path); len(matches) == 2 {
feedURL := fmt.Sprintf(`https://www.youtube.com/feeds/videos.xml?channel_id=%s`, matches[1])
if _, channelID, found := strings.Cut(decodedUrl.Path, "channel/"); found {
feedURL := "https://www.youtube.com/feeds/videos.xml?channel_id=" + channelID
return Subscriptions{NewSubscription(websiteURL, feedURL, parser.FormatAtom)}, nil
}
@@ -321,7 +315,7 @@ func (f *SubscriptionFinder) FindSubscriptionsFromYouTubePlaylistPage(websiteURL
if (strings.HasPrefix(decodedUrl.Path, "/watch") && decodedUrl.Query().Has("list")) || strings.HasPrefix(decodedUrl.Path, "/playlist") {
playlistID := decodedUrl.Query().Get("list")
feedURL := fmt.Sprintf(`https://www.youtube.com/feeds/videos.xml?playlist_id=%s`, playlistID)
feedURL := "https://www.youtube.com/feeds/videos.xml?playlist_id=" + playlistID
return Subscriptions{NewSubscription(websiteURL, feedURL, parser.FormatAtom)}, nil
}
+45 -22
View File
@@ -8,8 +8,6 @@ import (
"errors"
"fmt"
"log/slog"
"slices"
"strings"
"time"
"miniflux.app/v2/internal/crypto"
@@ -78,12 +76,20 @@ func (s *Storage) UpdateEntryTitleAndContent(entry *model.Entry) error {
title=$1,
content=$2,
reading_time=$3,
document_vectors = setweight(to_tsvector(left(coalesce($1, ''), 500000)), 'A') || setweight(to_tsvector(left(coalesce($2, ''), 500000)), 'B')
document_vectors = setweight(to_tsvector($4), 'A') || setweight(to_tsvector($5), 'B')
WHERE
id=$4 AND user_id=$5
id=$6 AND user_id=$7
`
if _, err := s.db.Exec(query, entry.Title, entry.Content, entry.ReadingTime, entry.ID, entry.UserID); err != nil {
if _, err := s.db.Exec(
query,
entry.Title,
entry.Content,
entry.ReadingTime,
truncateStringForTSVectorField(entry.Title),
truncateStringForTSVectorField(entry.Content),
entry.ID,
entry.UserID); err != nil {
return fmt.Errorf(`store: unable to update entry #%d: %v`, entry.ID, err)
}
@@ -122,8 +128,8 @@ func (s *Storage) createEntry(tx *sql.Tx, entry *model.Entry) error {
$9,
$10,
now(),
setweight(to_tsvector(left(coalesce($1, ''), 500000)), 'A') || setweight(to_tsvector(left(coalesce($6, ''), 500000)), 'B'),
$11
setweight(to_tsvector($11), 'A') || setweight(to_tsvector($12), 'B'),
$13
)
RETURNING
id, status, created_at, changed_at
@@ -140,7 +146,9 @@ func (s *Storage) createEntry(tx *sql.Tx, entry *model.Entry) error {
entry.UserID,
entry.FeedID,
entry.ReadingTime,
pq.Array(removeEmpty(removeDuplicates(entry.Tags))),
truncateStringForTSVectorField(entry.Title),
truncateStringForTSVectorField(entry.Content),
pq.Array(entry.Tags),
).Scan(
&entry.ID,
&entry.Status,
@@ -178,10 +186,10 @@ func (s *Storage) updateEntry(tx *sql.Tx, entry *model.Entry) error {
content=$4,
author=$5,
reading_time=$6,
document_vectors = setweight(to_tsvector(left(coalesce($1, ''), 500000)), 'A') || setweight(to_tsvector(left(coalesce($4, ''), 500000)), 'B'),
tags=$10
document_vectors = setweight(to_tsvector($7), 'A') || setweight(to_tsvector($8), 'B'),
tags=$12
WHERE
user_id=$7 AND feed_id=$8 AND hash=$9
user_id=$9 AND feed_id=$10 AND hash=$11
RETURNING
id
`
@@ -193,10 +201,12 @@ func (s *Storage) updateEntry(tx *sql.Tx, entry *model.Entry) error {
entry.Content,
entry.Author,
entry.ReadingTime,
truncateStringForTSVectorField(entry.Title),
truncateStringForTSVectorField(entry.Content),
entry.UserID,
entry.FeedID,
entry.Hash,
pq.Array(removeEmpty(removeDuplicates(entry.Tags))),
pq.Array(entry.Tags),
).Scan(&entry.ID)
if err != nil {
@@ -628,17 +638,30 @@ func (s *Storage) UnshareEntry(userID int64, entryID int64) (err error) {
return
}
func removeDuplicates(l []string) []string {
slices.Sort(l)
return slices.Compact(l)
}
// truncateStringForTSVectorField truncates a string to fit within the maximum size for a TSVector field in PostgreSQL.
func truncateStringForTSVectorField(s string) string {
// The length of a tsvector (lexemes + positions) must be less than 1 megabyte.
const maxTSVectorSize = 1024 * 1024
func removeEmpty(l []string) []string {
var finalSlice []string
for _, item := range l {
if strings.TrimSpace(item) != "" {
finalSlice = append(finalSlice, item)
if len(s) < maxTSVectorSize {
return s
}
// Truncate to fit under the limit, ensuring we don't break UTF-8 characters
truncated := s[:maxTSVectorSize-1]
// Walk backwards to find the last complete UTF-8 character
for i := len(truncated) - 1; i >= 0; i-- {
if (truncated[i] & 0x80) == 0 {
// ASCII character, we can stop here
return truncated[:i+1]
}
if (truncated[i] & 0xC0) == 0xC0 {
// Start of a multi-byte UTF-8 character
return truncated[:i]
}
}
return finalSlice
// Fallback: return empty string if we can't find a valid UTF-8 boundary
return ""
}
+96
View File
@@ -0,0 +1,96 @@
// SPDX-FileCopyrightText: Copyright The Miniflux Authors. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package storage
import (
"strings"
"testing"
)
func TestTruncateStringForTSVectorField(t *testing.T) {
// Test case 1: Short Chinese text should not be truncated
shortText := "这是一个简短的中文测试文本"
result := truncateStringForTSVectorField(shortText)
if result != shortText {
t.Errorf("Short text should not be truncated, got %s", result)
}
// Test case 2: Long Chinese text should be truncated to stay under 1MB
// Generate a long Chinese string that would exceed 1MB
const megabyte = 1024 * 1024
chineseChar := "汉"
longText := strings.Repeat(chineseChar, megabyte/len(chineseChar)+1000) // Ensure it exceeds 1MB
result = truncateStringForTSVectorField(longText)
// Verify the result is under 1MB
if len(result) >= megabyte {
t.Errorf("Truncated text should be under 1MB, got %d bytes", len(result))
}
// Verify the result is still valid UTF-8 and doesn't cut in the middle of a character
if !strings.HasPrefix(longText, result) {
t.Error("Truncated text should be a prefix of original text")
}
// Test case 3: Text exactly at limit should not be truncated
limitText := strings.Repeat("a", megabyte-1)
result = truncateStringForTSVectorField(limitText)
if result != limitText {
t.Error("Text under limit should not be truncated")
}
// Test case 4: Mixed Chinese and ASCII text
mixedText := strings.Repeat("测试Test汉字", megabyte/20) // Create large mixed text
result = truncateStringForTSVectorField(mixedText)
if len(result) >= megabyte {
t.Errorf("Mixed text should be truncated under 1MB, got %d bytes", len(result))
}
// Verify no broken UTF-8 sequences
if !strings.HasPrefix(mixedText, result) {
t.Error("Truncated mixed text should be a valid prefix")
}
// Test case 5: Large text ending with ASCII characters
asciiSuffix := strings.Repeat("a", megabyte-100) + strings.Repeat("测试", 50) + "abcdef"
result = truncateStringForTSVectorField(asciiSuffix)
if len(result) >= megabyte {
t.Errorf("ASCII suffix text should be truncated under 1MB, got %d bytes", len(result))
}
// Should end with ASCII character
if !strings.HasPrefix(asciiSuffix, result) {
t.Error("Truncated ASCII suffix text should be a valid prefix")
}
// Test case 6: Large ASCII text to cover ASCII branch in UTF-8 detection
largeAscii := strings.Repeat("abcdefghijklmnopqrstuvwxyz", megabyte/26+1000)
result = truncateStringForTSVectorField(largeAscii)
if len(result) >= megabyte {
t.Errorf("Large ASCII text should be truncated under 1MB, got %d bytes", len(result))
}
// Should be a prefix
if !strings.HasPrefix(largeAscii, result) {
t.Error("Truncated ASCII text should be a valid prefix")
}
// Test case 7: Edge case - string that would trigger the fallback
// Create a pathological case: all continuation bytes without start bytes
// This should trigger the fallback because there are no valid UTF-8 boundaries
invalidBytes := make([]byte, megabyte)
for i := range invalidBytes {
invalidBytes[i] = 0x80 // Continuation byte without start byte
}
result = truncateStringForTSVectorField(string(invalidBytes))
// Should return empty string as fallback
if result != "" {
t.Errorf("Invalid UTF-8 continuation bytes should return empty string, got %d bytes", len(result))
}
}
+3 -14
View File
@@ -101,7 +101,7 @@ func (e *Engine) ParseTemplates() error {
}
// Render process a template.
func (e *Engine) Render(name string, data map[string]interface{}) []byte {
func (e *Engine) Render(name string, data map[string]any) []byte {
tpl, ok := e.templates[name]
if !ok {
panic("This template does not exists: " + name)
@@ -114,19 +114,8 @@ func (e *Engine) Render(name string, data map[string]interface{}) []byte {
"elapsed": func(timezone string, t time.Time) string {
return elapsedTime(printer, timezone, t)
},
"t": func(key interface{}, args ...interface{}) string {
switch k := key.(type) {
case string:
return printer.Printf(k, args...)
case error:
return k.Error()
default:
return ""
}
},
"plural": func(key string, n int, args ...interface{}) string {
return printer.Plural(key, n, args...)
},
"t": printer.Printf,
"plural": printer.Plural,
})
var b bytes.Buffer
+3 -17
View File
@@ -34,7 +34,6 @@ func (f *funcMap) Map() template.FuncMap {
return template.FuncMap{
"formatFileSize": formatFileSize,
"dict": dict,
"hasKey": hasKey,
"truncate": truncate,
"isEmail": isEmail,
"baseURL": config.Opts.BaseURL,
@@ -77,9 +76,7 @@ func (f *funcMap) Map() template.FuncMap {
"mustBeProxyfied": func(mediaType string) bool {
return slices.Contains(config.Opts.MediaProxyResourceTypes(), mediaType)
},
"domain": urllib.Domain,
"hasPrefix": strings.HasPrefix,
"contains": strings.Contains,
"domain": urllib.Domain,
"replace": func(str, old, new string) string {
return strings.Replace(str, old, new, 1)
},
@@ -132,20 +129,9 @@ func dict(values ...interface{}) (map[string]interface{}, error) {
return dict, nil
}
func hasKey(dict map[string]string, key string) bool {
if value, found := dict[key]; found {
return value != ""
}
return false
}
func truncate(str string, max int) string {
runes := 0
for i := range str {
runes++
if runes > max {
return str[:i] + "…"
}
if runes := []rune(str); len(runes) > max {
return string(runes[:max]) + "…"
}
return str
}
-12
View File
@@ -43,18 +43,6 @@ func TestDictWithInvalidMap(t *testing.T) {
}
}
func TestHasKey(t *testing.T) {
input := map[string]string{"k": "v"}
if !hasKey(input, "k") {
t.Fatal(`This key exists in the map and should returns true`)
}
if hasKey(input, "missing") {
t.Fatal(`This key doesn't exists in the given map and should returns false`)
}
}
func TestTruncateWithShortTexts(t *testing.T) {
scenarios := []string{"Short text", "Короткий текст"}
+5 -5
View File
@@ -40,13 +40,13 @@ func (m *middleware) handleUserSession(next http.Handler) http.Handler {
next.ServeHTTP(w, r)
} else {
slog.Debug("Redirecting to login page because no user session has been found",
slog.Any("url", r.RequestURI),
slog.String("url", r.RequestURI),
)
html.Redirect(w, r, route.Path(m.router, "login"))
}
} else {
slog.Debug("User session found",
slog.Any("url", r.RequestURI),
slog.String("url", r.RequestURI),
slog.Int64("user_id", session.UserID),
slog.Int64("user_session_id", session.ID),
)
@@ -102,7 +102,7 @@ func (m *middleware) handleAppSession(next http.Handler) http.Handler {
if !crypto.ConstantTimeCmp(session.Data.CSRF, formValue) && !crypto.ConstantTimeCmp(session.Data.CSRF, headerValue) {
slog.Warn("Invalid or missing CSRF token",
slog.Any("url", r.RequestURI),
slog.String("url", r.RequestURI),
slog.String("form_csrf", formValue),
slog.String("header_csrf", headerValue),
)
@@ -141,7 +141,7 @@ func (m *middleware) getAppSessionValueFromCookie(r *http.Request) *model.Sessio
session, err := m.store.AppSession(cookieValue)
if err != nil {
slog.Debug("Unable to fetch app session from the database; another session will be created",
slog.Any("cookie_value", cookieValue),
slog.String("cookie_value", cookieValue),
slog.Any("error", err),
)
return nil
@@ -185,7 +185,7 @@ func (m *middleware) getUserSessionFromCookie(r *http.Request) *model.UserSessio
session, err := m.store.UserSessionByToken(cookieValue)
if err != nil {
slog.Error("Unable to fetch user session from the database",
slog.Any("cookie_value", cookieValue),
slog.String("cookie_value", cookieValue),
slog.Any("error", err),
)
return nil
-10
View File
@@ -5,8 +5,6 @@ package ui // import "miniflux.app/v2/internal/ui"
import (
"net/http"
"regexp"
"strings"
"miniflux.app/v2/internal/http/request"
"miniflux.app/v2/internal/http/response/html"
@@ -61,14 +59,6 @@ func (h *handler) updateSettings(w http.ResponseWriter, r *http.Request) {
view.Set("countWebAuthnCerts", h.store.CountWebAuthnCredentialsByUserID(loggedUser.ID))
view.Set("webAuthnCerts", creds)
// Sanitize the end of the block & Keep rules
cleanEnd := regexp.MustCompile(`(?m)\r\n\s*$`)
settingsForm.BlockFilterEntryRules = cleanEnd.ReplaceAllLiteralString(settingsForm.BlockFilterEntryRules, "")
settingsForm.KeepFilterEntryRules = cleanEnd.ReplaceAllLiteralString(settingsForm.KeepFilterEntryRules, "")
// Clean carriage returns for Windows environments
settingsForm.BlockFilterEntryRules = strings.ReplaceAll(settingsForm.BlockFilterEntryRules, "\r\n", "\n")
settingsForm.KeepFilterEntryRules = strings.ReplaceAll(settingsForm.KeepFilterEntryRules, "\r\n", "\n")
if validationErr := settingsForm.Validate(); validationErr != nil {
view.Set("errorMessage", validationErr.Translate(loggedUser.Language))
html.OK(w, r, view.Render("settings"))
+1 -1
View File
@@ -145,7 +145,7 @@ a:hover {
/* Page header and footer*/
.page-header {
padding-inline: 3px;
margin-bottom: 25px;
margin-bottom: 10px;
}
.page-footer {
+18 -6
View File
@@ -18,22 +18,34 @@ func IsAbsoluteURL(link string) bool {
return u.IsAbs()
}
// AbsoluteURL converts the input URL as absolute URL if necessary.
func AbsoluteURL(baseURL, input string) (string, error) {
// GetAbsoluteURL return the absolute form of `input` is possible, as well as its parser form.
func GetAbsoluteURL(input string) (string, *url.URL, error) {
if strings.HasPrefix(input, "//") {
return "https:" + input, nil
return "https:" + input, nil, nil
}
if strings.HasPrefix(input, "https://") || strings.HasPrefix(input, "http://") {
return input, nil
return input, nil, nil
}
u, err := url.Parse(input)
if err != nil {
return "", fmt.Errorf("unable to parse input URL: %v", err)
return "", nil, fmt.Errorf("unable to parse input URL: %v", err)
}
if u.IsAbs() {
return u.String(), nil
return u.String(), u, nil
}
return "", u, nil
}
// AbsoluteURL converts the input URL as absolute URL if necessary.
func AbsoluteURL(baseURL, input string) (string, error) {
absURL, u, err := GetAbsoluteURL(input)
if err != nil {
return "", err
}
if absURL != "" {
return absURL, nil
}
base, err := url.Parse(baseURL)
+3 -1
View File
@@ -1,5 +1,5 @@
.\" Manpage for miniflux.
.TH "MINIFLUX" "1" "June 15, 2025" "\ \&" "\ \&"
.TH "MINIFLUX" "1" "June 23, 2025" "\ \&" "\ \&"
.SH NAME
miniflux \- Minimalist and opinionated feed reader
@@ -344,6 +344,8 @@ Default is empty\&.
.B LISTEN_ADDR
Address to listen on. Use absolute path to listen on Unix socket (/var/run/miniflux.sock)\&.
.br
Multiple addresses can be specified, separated by commas. For example: 127.0.0.1:8080, 127.0.0.1:8081\&.
.br
Default is 127.0.0.1:8080\&.
.TP
.B LOG_DATE_TIME
+1 -1
View File
@@ -1,4 +1,4 @@
FROM docker.io/library/golang:alpine3.20 AS build
FROM docker.io/library/golang:alpine3.22 AS build
RUN apk add --no-cache build-base git make
ADD . /go/src/app
WORKDIR /go/src/app