1berry/wireshark
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
wireshark/test/conftest.py

34 lines
935 B
Python
Raw Permalink Normal View History

Tests: add minimal pytest support pytest is a powerful test framework. The initial motivation is to have much better detailed reporting when a test fails, see for example https://docs.pytest.org/en/latest/ Additionally, it has full parallelization support at the test level (via the pytest-xdist plugin) instead of being limited to the suite level (which currently has to be hard-coded via CMakeLists.txt). Usage with the build dir in /tmp/wsbuild and src dir in /tmp/wireshark: export WS_BIN_PATH=/tmp/wsbuild/run pytest /tmp/wireshark/tests For parallelization support and verbose printing: pip install pytest-xdist pytest -nauto -v /tmp/wireshark/tests To limit yourself to a case based on a pattern: pytest -nauto -v /tmp/wireshark/tests -k test_unit_ctest_coverage Tested on Arch Linux with Python 3.6.5, pytest-3.6.2, xdist-1.22.2. pytest -n8 finished in 82 seconds while ctest -j8 required 87 seconds. Change-Id: I832f4dd9f988d6656df795327e81610accf54b9f Reviewed-on: https://code.wireshark.org/review/28651 Reviewed-by: Gerald Combs <gerald@wireshark.org> Reviewed-by: Anders Broman <a.broman58@gmail.com>
2018-07-06 17:34:01 +02:00
#
# Wireshark tests
#
# Copyright (c) 2018 Peter Wu <peter@lekensteyn.nl>
#
# SPDX-License-Identifier: GPL-2.0-or-later
#
test: allow running pytest without specifying the tests directory This allows `pytest` to be executed from the top-level source or build directory (or any other directory below). Change-Id: Ib7af2ea2aaf01319d6839d2dc67228fbb5a7bc34 Reviewed-on: https://code.wireshark.org/review/31370 Petri-Dish: Peter Wu <peter@lekensteyn.nl> Tested-by: Petri Dish Buildbot Reviewed-by: Anders Broman <a.broman58@gmail.com>
2019-01-03 23:43:56 +01:00
'''pytest configuration'''
Tests: add minimal pytest support pytest is a powerful test framework. The initial motivation is to have much better detailed reporting when a test fails, see for example https://docs.pytest.org/en/latest/ Additionally, it has full parallelization support at the test level (via the pytest-xdist plugin) instead of being limited to the suite level (which currently has to be hard-coded via CMakeLists.txt). Usage with the build dir in /tmp/wsbuild and src dir in /tmp/wireshark: export WS_BIN_PATH=/tmp/wsbuild/run pytest /tmp/wireshark/tests For parallelization support and verbose printing: pip install pytest-xdist pytest -nauto -v /tmp/wireshark/tests To limit yourself to a case based on a pattern: pytest -nauto -v /tmp/wireshark/tests -k test_unit_ctest_coverage Tested on Arch Linux with Python 3.6.5, pytest-3.6.2, xdist-1.22.2. pytest -n8 finished in 82 seconds while ctest -j8 required 87 seconds. Change-Id: I832f4dd9f988d6656df795327e81610accf54b9f Reviewed-on: https://code.wireshark.org/review/28651 Reviewed-by: Gerald Combs <gerald@wireshark.org> Reviewed-by: Anders Broman <a.broman58@gmail.com>
2018-07-06 17:34:01 +02:00
test: convert capture tests to use fixtures, fix tests without dumpcap Add a new --capture-interface option to pytest, similar to test.py. It will grab some Ethernet interface on Windows. An empty value overrides this and disables capture tests. Remove the test.py --enable-capture option since that is implied by the --capture-interface option. Port the `test.py --program-path` option to pytest and additionally make the pytest look in the current working directory if neither WS_BIN_PATH nor --program-path are specified. Drop config.setProgramPath, this allows tests to be run even if not all binaries are available. With all capture tests converted to fixtures, it is now possible to run tests when Wireshark is not built with libpcap as tests that depend on cmd_dumpcap (or capture_interface) will be skipped. Bug: 14949 Change-Id: Ie802c07904936de4cd30a4c68b6a5139e6680fbd Reviewed-on: https://code.wireshark.org/review/30656 Petri-Dish: Peter Wu <peter@lekensteyn.nl> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2018-11-15 18:44:59 +01:00
def pytest_addoption(parser):
parser.addoption('--disable-capture', action='store_true',
help='Disable capture tests'
)
GitLab CI+Tests: Add a --disable-gui option Add a --disable-gui option and use it to skip GUI tests in the Windows Build job.
2023-09-09 18:37:36 -07:00
parser.addoption('--disable-gui', action='store_true',
help='Disable GUI tests'
)
Tests: Fix default option argument
2023-09-24 13:14:20 +01:00
parser.addoption('--build-type', default='RelWithDebInfo',
GitLab CI: Use debug build type with merge requests Try to run our merge requests in Debug mode to make sure the test suite has the best chance of catching any issues. This means we no longer have a Release build. This build was intended to catch warnings that only appears with -O3 optimization level. It seems to have been successful at that. If that becomes an issue again we may want to add a new job with a Release type build. This should help catch more runtime errors and assertions that sometimes slip through the cracks (and may also be related with recent changes to the build configuration) and should be easily caught in the tests.
2023-09-23 10:35:36 +01:00
help='CMake build type for multi-config generators.'
)
parser.addoption('--program-path',
help='Path to Wireshark executables.'
)
test: fail tests when programs are missing Building only a subset of programs is not a very common situation, it is more likely that some feature was accidentally disabled. For that reason, fail tests by default unless a program is explicitly permitted to be missing. The '-v' test is now dropped from the Travis tests, the sole reason of adding it was to see which tests got (accidentally) skipped. Change-Id: I725f4508541d8ed980e17d69fb7aee1ad2875d73 Reviewed-on: https://code.wireshark.org/review/31660 Petri-Dish: Peter Wu <peter@lekensteyn.nl> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2019-01-21 15:05:37 +01:00
parser.addoption('--skip-missing-programs',
help='Skip tests that lack programs from this list instead of failing'
GitLab CI: Use debug build type with merge requests Try to run our merge requests in Debug mode to make sure the test suite has the best chance of catching any issues. This means we no longer have a Release build. This build was intended to catch warnings that only appears with -O3 optimization level. It seems to have been successful at that. If that becomes an issue again we may want to add a new job with a Release type build. This should help catch more runtime errors and assertions that sometimes slip through the cracks (and may also be related with recent changes to the build configuration) and should be easily caught in the tests.
2023-09-23 10:35:36 +01:00
' them. Use "all" to ignore all missing programs.'
)
Tests: Fix --enable-release option
2023-06-04 17:43:24 +01:00
parser.addoption('--enable-release', action='store_true',
help='Enable release tests'
)
test: convert capture tests to use fixtures, fix tests without dumpcap Add a new --capture-interface option to pytest, similar to test.py. It will grab some Ethernet interface on Windows. An empty value overrides this and disables capture tests. Remove the test.py --enable-capture option since that is implied by the --capture-interface option. Port the `test.py --program-path` option to pytest and additionally make the pytest look in the current working directory if neither WS_BIN_PATH nor --program-path are specified. Drop config.setProgramPath, this allows tests to be run even if not all binaries are available. With all capture tests converted to fixtures, it is now possible to run tests when Wireshark is not built with libpcap as tests that depend on cmd_dumpcap (or capture_interface) will be skipped. Bug: 14949 Change-Id: Ie802c07904936de4cd30a4c68b6a5139e6680fbd Reviewed-on: https://code.wireshark.org/review/30656 Petri-Dish: Peter Wu <peter@lekensteyn.nl> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2018-11-15 18:44:59 +01:00
test: make it possible to use pytest-style test fixtures Currently all binaries must be available or no tests will be executed. This is inconvenient if you just want to test a single binary (e.g. text2pcap) without having to build epan. The problem is essentially that tests lack dependency annotations. To solve this problem, add the required dependencies as parameters to each test (so-called 'fixtures' in pytest). Skip a test if a binary (such as tshark) is unavailable. As a demonstration, suite_dissection.py is converted. Over time, tests should no longer depend on config.py due to explicit dependencies fixtures (listed in fixtures_ws.py). Since the unittest module does not support such dependency injections, create a small glue for use with pytest and an (incomplete) emulation layer for use with test.py. Tested with pytest 3.8.2 + Python 3.7.0 and pytest 3.0.3 + Python 3.4.3. Python 2.7 is not supported and will fail. Test commands: ~/wireshark/test/test.py -p ~/build/run WS_BIN_PATH=~/build/run pytest ~/wireshark/test -ra Change-Id: I6dc8c28f5c8b7bbc8f4c04838e9bf085cd22eb0b Ping-Bug: 14949 Reviewed-on: https://code.wireshark.org/review/30220 Tested-by: Petri Dish Buildbot Petri-Dish: Peter Wu <peter@lekensteyn.nl> Reviewed-by: Anders Broman <a.broman58@gmail.com>
2018-10-15 16:07:30 +02:00
from fixtures_ws import *
test: convert capture tests to use fixtures, fix tests without dumpcap Add a new --capture-interface option to pytest, similar to test.py. It will grab some Ethernet interface on Windows. An empty value overrides this and disables capture tests. Remove the test.py --enable-capture option since that is implied by the --capture-interface option. Port the `test.py --program-path` option to pytest and additionally make the pytest look in the current working directory if neither WS_BIN_PATH nor --program-path are specified. Drop config.setProgramPath, this allows tests to be run even if not all binaries are available. With all capture tests converted to fixtures, it is now possible to run tests when Wireshark is not built with libpcap as tests that depend on cmd_dumpcap (or capture_interface) will be skipped. Bug: 14949 Change-Id: Ie802c07904936de4cd30a4c68b6a5139e6680fbd Reviewed-on: https://code.wireshark.org/review/30656 Petri-Dish: Peter Wu <peter@lekensteyn.nl> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2018-11-15 18:44:59 +01:00
Reference in New Issue Copy Permalink