Py Professional Edition, previously covered in a blog post about using Behave and an older blog post about BDD. This tutorial puts pytest-bdd to use.
Want the finished code? It’s in available in a GitHub repo.
What and Why for BDD
If you’re new to BDD, it can appear a bit strange. You write a programming-free specification, known as a “feature”, that describes how the software is supposed to behave. You then implement that specification with test code.
This two-step process shines in several workflows:
- Large teams with “test engineers”, business analysts or/and domain experts. That is, those who understand business and can read English but not Python code.
- Complex software in which you want to think about what’s happening in a simpler fashion than test code
BDD can seem like overhead. But if you find yourself lost in your test code, the feature spec can help you step back from the details and follow the high-level intent. Even better, when you come back to your code months later, it is much easier to get up-to-date. Finally, if you feel committed to the “Given/When/Then” test style, pytest-bdd moves that commitment from “dating” to “marriage”.
Setup
We’ll use the same example application as the previous blog post.
To follow along at home, make sure you have Python 3.7 (our example uses dataclasses) and Pipenv installed. Clone the repo at and make sure Pipenv has created an interpreter for you. (You can do both of those steps from within PyCharm.) Make sure Pipenv has installed
pytest and
pytest-bdd into your interpreter.
Then, open the directory in PyCharm and make sure you have set the
Python Integrated Tools -> Default Test Runner to
pytest, as done in the previous blog post.
This time, though, we also need to set
Languages & Frameworks -> BDD -> Preferred BD framework to
pytest-bdd. This will enable many of the features discussed in this tutorial. More information is available in the PyCharm Help on pytest-bdd.
Now you’re ready to follow the material below. Right-click on the tests directory and choose
Run ‘pytest in tests’. If all the tests pass correctly, you’re setup.
Let’s Make a Feature
What parts of our project deliver business value? We’re going to take these requirements, as “features”, and write them in feature files. These features are specified using a subset of the Gherkin language.
Right click on the
tests folder and select
New, then choose
Gherkin feature file.
Name the file
games. Note: Some like to group BDD tests under tests/features.
Here’s the default file contents generated by PyCharm:
# Created by pauleveritt at 8/6/18 Feature: #Enter feature name here # Enter feature description here Scenario: # Enter scenario name here # Enter steps here
Let’s change the feature file’s specification to say the following:
Feature: Games laxleague games are between two teams and have a score resulting in a winner or a tie. Scenario: Determine The Winner Given a home team of Blue And a visiting team of Red When the score is 5 for Blue to 4 for Red Then Blue is the winner
This Scenario shows the basics of BDD:
- Each Feature can have multiple Scenarios
- Each Scenario has multiple steps
- The steps revolve around the “given/when/then” approach to test specification
- Step types can be continued with the
Andkeyword
Givenspecifies inputs
Whenspecifies the logic being tested
Thenspecifies the result
We might have a number of other scenarios relating to games. Specifically, the “winner” when the score is tied. For now, this provides enough to implement.
As we type this feature in, we see some of the features PyCharm Professional provides in its Gherkin support:
- Syntax highlighting of Gherkin keywords
- Reformat Code fixes indentation
- Autocomplete on keywords
- Warnings on unimplemented steps
Implement The Steps
If you run the tests, you’ll see….no difference. The
games.feature file isn’t a test: it’s a specification. We need to implement, in test code, each of the scenarios and scenario steps.
PyCharm can help on this. As mentioned above, PyCharm warns you about an “Undefined step reference”:
If you Alt-Enter on the warnings, PyCharm will offer to either
Create step definition or
Create all steps definition. Since 2018.2 does the latter in an unexpected way (note: it’s being worked on), let’s choose the former, and provide a
File name: of
test_games_feature:
Here’s what the generated test file
test_games_feature.py looks like:
from pytest_bdd import scenario, given, when, then @given("a home team of Blue") def step_impl(): raise NotImplementedError(u'STEP: Given a home team of Blue')
These are just stubs, of course, which we’ll have to come back and name/implement. We could implement the other steps by hand. Let’s instead let PyCharm continue generating the stubs, into the same file.
2018.2 doesn’t generate the scenario, which is what actually triggers the running of the test. Let’s provide the scenario, as well as implement each step, resulting in the following for the
test_games_feature.py implementation:
from pytest_bdd import scenario, given, when, then from laxleague.games import Game from laxleague.teams import Team @scenario('games.feature', 'Determine The Winner') def test_games_feature(): pass @given('a home team of Blue') def blue(): return Team('Blue') @given('a visiting team of Red') def red(): return Team('Red') @given('a game between them') def game_red_blue(blue, red): return Game(home_team=blue, visitor_team=red) @when('the score is 10 for Blue to 5 for Red') def record_score(game_red_blue): game_red_blue.record_score(10, 5) @then('Blue is the winner') def winner(game_red_blue): assert 'Blue' == game_red_blue.winner.name
To see this in action, let’s run the test then take a look at what pytest-bdd is doing.
Run the Tests
You’ve already run your regular pytest tests, with fixtures and the like. What extra does it take to also run your pytest-bdd tests? Nothing! Just run your tests:
Your pytest-bdd tests show up just like any other test.
Let’s take a look at some things pytest-bdd is doing:
- The
@scenariodecorated function
test_games_featureis the only function in the file with the
test_prefix. That means, this file only has one test. And guess what? The function itself doesn’t do anything. It’s just a marker.
- We need two teams, so we implement the two Given steps by making a Blue and Red team.
- We also need a game between these two teams. This is the third step in our
games.featurescenario. Not that this function takes two arguments. As it turns out, pytest-bdd steps are pytest fixtures which can be injected into the functions for each step. (In fact, any pytest fixture can be injected.)
- Now that we have a game setup, we use @when to run the logic being tested by recording a score.
- Did our logic work? We use
@thento do our assertion. This is where our test passes or fails.
It’s an interesting approach. It’s verbose, but it clearly delineates the “Given/When/Then” triad of good test cases. Plus, you can read the almost-human-language Gherkin file to quickly understand what’s the behavior being tested.
Test Parameters
You might have noticed that the feature file specified a score for the game but it was ignored in the implemented tests. pytest-bdd has a feature where you can extract parameters from strings. It has several parsing schemes. We’ll use the simplest, starting by adding
parsers to the import from
pytest_bdd:
from pytest_bdd import scenario, given, when, then, parsers
Note: You could also do this the productive way by using the symbol and letting PyCharm generate the import for you.
The feature file says this:
When the score is 10 for Blue to 5 for Red
Let’s change our
@when decorator to parse out the score:
@when(parsers.parse('the score is {home:d} for Blue to {visitor:d} for Red'))
When we do so, PyCharm warns us that
Not all arguments.... were used. You can type them in, but PyCharm knows how to do it. Hit
Alt-Enter and accept the first item:
After changing
record_score to use these values, our function looks like this:
@when(parsers.parse('the score is {home:d} for Blue to {visitor:d} for Red')) def record_score(game_red_blue, home, visitor): game_red_blue.record_score(home, visitor)
As we experiment with lots of combinations, moving this to the feature file is very helpful, particularly for test engineers.
Want more? Gherkin and pytest-bdd support “Scenario Outlines” where you can batch parameterize your inputs and outputs, just like we saw previously with
pytest.mark.parametrize.
Use pytest Fixtures and Features
As noted above, pytest-bdd treats scenario steps as pytest fixtures. But any pytest fixture can be used, which makes pytest-bdd a powerful approach for BDD.
For example, imagine we have an indirection where there are different types of Game implementations. We’d like to put the choice of which class is used behind a fixture, in
conftest.py:
@pytest.fixture def game_type(): return Game
We can then use this fixture in the code>@given step when we instantiate a Game:
@given('a game between them') def game_red_blue(game_type, blue, red): return game_type(home_team=blue, visitor_team=red)
Productive BDD
Prior to 2018.2, PyCharm provided BDD for Behave, but nothing for pytest-bdd. Admittedly, there are still loose ends being worked on. Still, what’s available now is quite productive, with IDE features brought to bear when getting in the pytest-bdd flow.
Warnings
Forgot to implement a step? Typo in your
@scenario text identifier? PyCharm flags these with the (configurable) warning and error infrastructure.
Autocomplete
Obviously we autocomplete keywords from Gherkin and pytest-bdd. And as expected, we autocomplete function names as fixture parameters. But we also autocomplete strings from the feature file…for example, to fix the kind of error just mentioned for the scenario identifier. Also, parameters parsed out of strings can autocomplete.
Quick Info
Wonder what is a particular symbol and where it came from? You can use Quick Info or on-hover type information with Cmd-hover to see more information:
Conclusion
Doing behavior-driven development is a very different approach to testing. With pytest-bdd, you can keep much of what you know and use from the very-active pytest ecosystem when doing BDD. PyCharm Professional 2018.2 provides a “visual testing” frontend to keep you in a productive BDD flow.
12 Responses to PyCharm and pytest-bdd
Juan Burgos says:August 9, 2018
Perhaps you should parametrize too the @then to have a more “full working” tutorial.
Paul Everitt says:August 9, 2018
I stopped short on going too far into parameters. Probably deserves a follow-up (e.g. scenario outline.)
J Tan says:August 15, 2018
Would be nice to have an article in the future comparing the pros and cons between pytest-bdd and behave.
Paul Everitt says:August 15, 2018
I think the big point is, access to the pytest ecosystem.
Gaëtan de Menten says:August 20, 2018
Nice article but naming your function game_blue_red instead of game_red_blue would make things a lot less confusing/error prone.
Ken says:February 27, 2019
Can we debug with pytest_bdd? I can set breakpoint but I cannot evaluate the vars and expressions.
e.g. when I evaluate `game_red_blue.winner.name`, it spit out NameError –
Wojtek says:May 7, 2019
How about rewriting asserts and using pycharm’s pytest test runner to actually see the rewritten asserts?
Navya says:April 27, 2020
Hi,
I want to run one pytest file before running another pytest file.how ca we implement fixtures in second pytest file?
Paul Everitt says:April 28, 2020
It’s normal pytest fixture stuff. If you put the decorator on the fixture, then use that fixture name as an argument in a test, pytest will execute the fixture and pass in the return value.
Gowtham says:June 10, 2020
Is this pytest bdd support pycharm community version?
Roberto Pesce says:June 12, 2020
Hi, behavior-driven-development support is part of PyCharm Professional Edition
Sharon says:June 23, 2020
Hi, To run individual scenarios from the feature files in pycharm, which run-configuration template should we use ? i see the configuration templates available for behave and lettuce but not for pytest-bdd.