Story Reporters

Reporting is an essential element of BDD as it allows to monitor the outcome of the stories that have been run. At the heart of JBehave's reporting is the StoryReporter, which is used to report events as they occur.

Currently, the story reporters supported are:

  • PrintStreamOutput: generic reporter to a print stream, using formats that are configurable
  • ConsoleOutput: specialises the PrintStreamOutput to use TXT console output
  • TxtOutput: specialises the PrintStreamOutput to use TXT file output
  • HtmlOutput: specialises the PrintStreamOutput to use HTML file output
  • XmlOutput: specialises the PrintStreamOutput to use XML file output
  • PostStoryStatisticsCollector: collects statistics and stores them as properties after story is run
  • DelegatingStoryReporter: delegates to any number of reporters as a proxy.

Let's see an example of how we can configure the story reporters. We use the DelegatingStoryReporter as a proxy for the reporters we want to configure, in the example below a plain text reporter (both to System.out and to a file) and an HTML reporter to a file:


    public TraderStory() {
        // start with default story configuration, overriding story definer and reporter
        StoryConfiguration storyConfiguration = new MostUsefulStoryConfiguration();
        storyConfiguration.useStoryPathResolver(new UnderscoredCamelCaseResolver(".story"));
        String storyPath = storyConfiguration.storyPathResolver().resolve(this.getClass());
        storyConfiguration.useStoryLoader(new LoadFromClasspath(this.getClass().getClassLoader()));
        storyConfiguration.useStoryReporter(new StoryReporterBuilder(new FilePrintStreamFactory(storyPath))
                .outputTo("target/jbehave-reports").outputAsAbsolute(true) // only required with Ant (cf note below)
                .withDefaultFormats() // add default formats 
                .withFormats(CONSOLE, TXT, HTML, XML)
                .build(storyPath));
        useConfiguration(storyConfiguration);

        // start with default steps configuration
        StepsConfiguration stepsConfiguration = new MostUsefulStepsConfiguration();
        addSteps(createSteps(stepsConfiguration));
    }

    protected CandidateSteps[] createSteps(StepsConfiguration configuration) {
        return new StepsFactory(configuration).createCandidateSteps(new TraderSteps(new TradingService()), new BeforeAfterSteps());
    }

For a fully-working example of reporting configuration you may consult the trader example. Note that the configuration only needs to be done in one place, i.e. the parent Scenario class, and not repeated in every Scenario class that maps to a textual scenario. In general, the specification of the output directory is optional. It is necessary when running with Ant as a workaround for Ant's classloading strategy which uses the Ant Project.class ProtectionDomain, which in turn leads to the $ANT_HOME/lib as the code source location. A better solution would be to override the hacky AntClassLoader behaviour, as also noted by Ant folks in the AntClassLoader#defineClassFromData method. A user-friendly way to do is being investigated.

Note that we use the StoryReporterBuilder to implement a builder pattern for file-based reporters via the FilePrintStreamFactory, in which we inject the StoryPathResolver to derive the report file names from the story class, using the same name resolution mechanism used for mapping Java classes and textual stories. So, e.g., if the story class is com.example.MyStory, we'll end up with file report outputs of the form: com.example.my_story.[format] (where format is any of txt,html,xml in the example above).

The file-based print stream factory for the story reporting requires the story class to be provided in order to derive from it the output file paths (with the appropriate format extension). If you want to migrate multiple stories to using file-based reporting, an intermediate stepping stone might be to provide as a story class the parent class in which you build the story reporter (TraderStory.class in the example), which will result in all reporting (for a given format) to be appended to the same file. Then gradually you can refactor your story classes to provide their class to the parent (as the example shows). Otherwise, you may want to keep output in a single file, the choice is yours.

The builder provides defaults for all the formats supported, but if the user needs to create a bespoke instance of a reporter for a given format, it can be easily done by overriding the default. E.g. to override the reporter for TXT format to use a ".text" extension (a possibly keywords for a different Locale):


        StoryReporter reporter = new StoryReporterBuilder(factory){
               public StoryReporter reporterFor(String storyPath, Format format){
			           FilePrintStreamFactory factory = new FilePrintStreamFactory(new StoryLocation(storyPath, ouputLocationClass));
                       switch (format) {
                           case TXT:
                               factory.useConfiguration(new FileConfiguration("text"));
                               return new PrintStreamOutput(factory.getPrintStream(),
                                            new Properties(), new LocalizedKeywords(Locale.ITALIAN), true);
                            default:
                               return super.reporterFor(format);
                       }
                   }

Report Rendering

The generation of the reports is only the first part of a complete HTML-based reporting solution. Next we need to render the reports, aggregating all the ones that have been configured and generated in a given output directory, and presenting a collective index view for all formats configured. Moreover, we need to style the view, both for HTML and non-HTML report formats.

The rendering is the responsibility of the ReportRenderer. JBehave provides an implementation (FreemarkerReportRenderer) based on Freemarker to allow a templateable and easily styleable way to render the report views.

The default resources required for the report rendering are bundled in the jbehave-core.jar (extract ftl/*.ftl, js/*.js and style/*.css) but can be overridden. The FTL files need to be the classpath for the FreemarkerReportRenderer to find them, while the look and feel resources (js/*.js and style/*.css files) need to be copied to the target/jbehave-reports/rendered directory (or wherever the index page has been rendered to) Also note that the default style makes use of images found in the jbehave-site-resources.jar.

Also, note that the report formats configured should match ones found in the Ant or Maven execution for the report rendering task or goal (c.f. running stories for examples).

Reporting Statistics

The story statistics report above is treated in the rendering slightly differently from other reports, in that the statistics are displayed on the index page, if available. To ensure they are always available the story reporter builder is configured to have stats as a default format (although the default formats need to be added to the builder).

Default Formats

Default formats allow users to define a builder that can be used in multiple configurations without having to repeat tediously all the formats required, if these are used consistently. By default, "stats" is the only default format (used for collecting reporting statistics). To modify simply override the method withDefaultFormats(). E.g. to add "txt" as a default format:

        new StoryReporterBuilder(factory){
               protected StoryReporterBuilder withDefaultFormats() {
                    return withFormats(STATS, TXT);
               }
        }

By default, JBehave outputs file reports to the directory jbehave-reports (relative to the story class code source location, e.g. the target directory in Maven), but this can be changed via the builder:

        new StoryReporterBuilder(factory).outputTo("my-reports")

In some cases, i.e. when using with Ant, the class code source location may not be properly set, or different from what is expected. In this case, one can tell the builder to treat the output directory as an absolute path:

        new StoryReporterBuilder(factory).outputTo("target/my-reports").outputAsAbsolute(true)

If the default formats or the output file directory are modified via the builder, then correspondingly we need to inform the report render of these changes. See running stories for details on how to configure these changes.

Next?

The trader example is configured to have a fully-working end-to-end use of JBehave reporting (be sure to read the instructions on running examples). After running the build simply open the page target/jbehave-reports/rendered/index.html