# Benchmark suite

We need to set up a benchmark suite for DOLFIN to track improvements (or regressions) in efficiency.

The benchmark suite will be run nightly on a dedicated server and the results summarized and plotted on fenics.org.

The benchmarks will also be backported (where possible) to earlier versions of DOLFIN.

Suggested categories and benchmarks:

1. la

la benchmarks should iterate over all backends and include the following benchmarks

1a. matrix-vector product

1b. gmres solve Poisson

1c. gmres + ilu solve Poisson

1d. gmres (or cg) + amg solve

1e-h. same as above in parallel

2. mesh

2a. reading mesh from file

2b. initializing topology (call to init)

2c. computing boundary

2d. iterating over entities

2e. mesh refinement

2f. mesh partitioning

2g. mesh intersection

3. fem

fem benchmarks should also iterate over all la backends

need timings for specific parts of assembly, in particular computing the dof map, tabulate_tensor, add, and apply

3a. serial assembly of Poisson with Lagrange elements degree 1, 2, 3, 4, 5

3b. serial assembly of DG Poisson formulation

3c-d. same as above in parallel

4. function

4a. interpolation / restriction of a Function

4b. interpoilation / restriction of an Expression

4c. evaluation at arbitrary points

5. ode

5a. initializing weights for q = 1, 2, 3, ..., 10

5b. solving Lorenz on [0, 200] fixed time step

5c. some mono-adaptive solve

5d. some multi-adaptive solve

6. misc

5a. jit compiler

5b. newton solver

## Blueprint information

- Status:
- Complete

- Approver:
- None

- Priority:
- High

- Drafter:
- None

- Direction:
- Needs approval

- Assignee:
- None

- Definition:
- Approved

- Series goal:
- None

- Implementation:
- Implemented

- Milestone target:
- 1.0-beta2

- Started by
- Garth Wells

- Completed by
- Anders Logg

### Related branches

### Related bugs

### Sprints

### Whiteboard

GNW: Should also compare C++ versus Python performance.

AL: Good point

GNW: Poisson only is not a good test since it doesn't show up issues which arise for more complicated forms.

AL: Good point, please add a (limited) number of relevant test cases

AL: The list above is just a first iteration, please help out to expand and organize the test suite

AL: The benchbot is up and running. Please help out with providing relevant benchmarks before Johannes adds the final touches.

AL: I consider this implemented now (although with a somewhat different set of benchmarks). Johannes has backported quite a few of the benchmarks to DOLFIN 0.7.3