diff options
author | Keith Maxwell <keith.maxwell@gmail.com> | 2020-05-11 21:43:07 +0100 |
---|---|---|
committer | Rasmus Thomsen <oss@cogitri.dev> | 2020-05-12 06:22:41 +0000 |
commit | 1a7facbcc19063e529d3a72fb719e223c95a76c3 (patch) | |
tree | 1ba4fb4a53fa29aa4f90ccff7c30594185d2a982 | |
parent | 470184667e77e61df74b4b3e46c644173b9df1aa (diff) | |
download | aports-1a7facbcc19063e529d3a72fb719e223c95a76c3.tar.bz2 aports-1a7facbcc19063e529d3a72fb719e223c95a76c3.tar.xz |
community/rstcheck: fix intermittent test failures
Add a patch to close the multiprocessing pool in line with the Python 3
documentation.
The patch introduced by this commit has been submitted upstream:
https://github.com/myint/rstcheck/pull/67
Testing inside `dabuild sh`, before this change we can't get above run 11:
```
$ cd src/rstcheck-3.3.1/
$ for i in $(seq 99) ; do printf '%3d' $i && ./rstcheck.py --recursive examples/good || break ; done ; echo
1 (stalled)
$ for i in $(seq 99) ; do printf '%3d' $i && ./rstcheck.py --recursive examples/good || break ; done ; echo
1 2 3 4 5 6 7 8 9 10 11 (stalled)
$ for i in $(seq 99) ; do printf '%3d' $i && ./rstcheck.py --recursive examples/good || break ; done ; echo
1 2 3 (stalled)
```
After this change, 99 runs finish three times in a row:
```
$ abuild deps unpack prepare build
$ cd src/rstcheck-3.3.1/
$ for i in $(seq 99) ; do printf '%3d' $i && ./rstcheck.py --recursive examples/good || break ; done ; echo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
$ for i in $(seq 99) ; do printf '%3d' $i && ./rstcheck.py --recursive examples/good || break ; done ; echo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
$ for i in $(seq 99) ; do printf '%3d' $i && ./rstcheck.py --recursive examples/good || break ; done ; echo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
```
-rw-r--r-- | community/rstcheck/02-Close-the-multiprocessing.pool-after-use.patch | 59 | ||||
-rw-r--r-- | community/rstcheck/APKBUILD | 9 |
2 files changed, 64 insertions, 4 deletions
diff --git a/community/rstcheck/02-Close-the-multiprocessing.pool-after-use.patch b/community/rstcheck/02-Close-the-multiprocessing.pool-after-use.patch new file mode 100644 index 0000000000..d0680ef23d --- /dev/null +++ b/community/rstcheck/02-Close-the-multiprocessing.pool-after-use.patch @@ -0,0 +1,59 @@ +https://github.com/myint/rstcheck/pull/67 + +From 5e6e995305f66246e8f98acd15de2c39a75efa5b Mon Sep 17 00:00:00 2001 +From: Keith Maxwell <keith.maxwell@gmail.com> +Date: Mon, 11 May 2020 20:53:14 +0100 +Subject: [PATCH] Close the multiprocessing.pool after use + +So that the tests do not hang intermittently + +When preparing to release Alpine Linux 3.12, the tests for this package +would hang intermittently. This problem was isolated to the tests in +test.bash that run rstcheck.py over multiple files. + +The documentation for multiprocessing.pool explains that: + +> Warning multiprocessing.pool objects have internal resources that need +> to be properly managed (like any other resource) by using the pool as +> a context manager or by calling close() and terminate() manually. +> Failure to do this can lead to the process hanging on finalization. + +> Note that is not correct to rely on the garbage colletor to destroy +> the pool as CPython does not assure that the finalizer of the pool +> will be called (see object.__del__() for more information). + +https://docs.python.org/3/library/multiprocessing.html#module-multiprocessing.pool + +Before this commit close() was not called on the multiprocessing pool; +after this commit close() is called. + +The change in this commit was tested in an Alpine Linux container: + + podman run -ti --rm -v $PWD:/srv:Z -w /srv alpine:edge sh + +By running: + + apk add alpine-sdk python3 py3-docutils py3-setuptools bash + python3 setup.py build + sed -i '1s|^#!/usr/bin/env python$|#!/usr/bin/python3|' rstcheck.py + python3 ./test_rstcheck.py + bash ./test.bash +--- + rstcheck.py | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/rstcheck.py b/rstcheck.py +index a13ec2b..959808a 100755 +--- a/rstcheck.py ++++ b/rstcheck.py +@@ -981,6 +981,7 @@ def main(): + except (IOError, UnicodeError) as exception: + output_message(exception) + status = 1 ++ pool.close() + + return status + +-- +2.26.2 + diff --git a/community/rstcheck/APKBUILD b/community/rstcheck/APKBUILD index 74399fe8c1..91ff6fecd0 100644 --- a/community/rstcheck/APKBUILD +++ b/community/rstcheck/APKBUILD @@ -2,7 +2,7 @@ # Maintainer: Keith Maxwell <keith.maxwell@gmail.com> pkgname=rstcheck pkgver=3.3.1 -pkgrel=2 +pkgrel=3 pkgdesc="Checks syntax of reStructuredText and code blocks nested within it" url="https://github.com/myint/rstcheck" arch="noarch" @@ -10,10 +10,10 @@ license="MIT" depends="python3 py3-docutils py3-setuptools" checkdepends="bash" # use github because the test suite isn't part of the pypi package: -source="${pkgname}-${pkgver}.tar.gz::https://github.com/myint/$pkgname/archive/v$pkgver.tar.gz +source="$pkgname-$pkgver.tar.gz::https://github.com/myint/rstcheck/archive/v$pkgver.tar.gz 01-python3-test-suite.patch + 02-Close-the-multiprocessing.pool-after-use.patch " -[ "$CARCH" = "ppc64le" ] && options="!check" # test suite livelocks on ppc64le build() { python3 setup.py build @@ -29,4 +29,5 @@ package() { } sha512sums="07431d6c4789e4c005c4bf4be66f627c390f57e56f25b4f3db8b392bc7b904ca93c3e26145ab344e9b78109d2054e5f36167ab40b1ab023f08c85d6ec12cafbc rstcheck-3.3.1.tar.gz -e2a86661595debc58e5cf919277afbd6b50abcd97ea6d5383294b9b346a4344d837927cd678738d0f01f01d450c5a99f84732f09aa696e19e588a732d9a4c773 01-python3-test-suite.patch" +e2a86661595debc58e5cf919277afbd6b50abcd97ea6d5383294b9b346a4344d837927cd678738d0f01f01d450c5a99f84732f09aa696e19e588a732d9a4c773 01-python3-test-suite.patch +4cdd8825fb2bb0e0522f57487b37cce9509ebd73a47257cece8044ffe78b32cfab9052c6506c031059ab2bcc02a4ac166c59df724cfdd55598e697ba03802cd5 02-Close-the-multiprocessing.pool-after-use.patch" |