FEM Workbench Testing - GSoC Proposal

How about “Consensus on API Documentation” or “Consensus on Source Documentation”?

Thanks for the response. I have to say that I don’t agree with the maintenance point you are making. In my experience it requires very little set up and maintenance work as long as tools are used for what they are intended. I suggested using two tools with the taught that it would be much easier than pushing one tool to its limits where the real time consuming work normally starts. With that said I of course agree that having one system that handles C++ and Python very well would be the best option.

I’d also like to repeat what I wrote earlier in the thread. The most important aspect is to produce a end result that is actually helpful to us developers and newcomers. If we fail to meet this goal it was all for nothing. And I don’t think that we’ll be able to reach this goal for Python by using doxygen.

Heads up: the discussion on frameworks regarding documenting the c++ and python sourcecode for FC is continuing in:
Discussion: Consensus on API Documentation (link)

Thanks! Can I help you migrate the posts in any way? :slight_smile:

I just committed stuff that makes it very easy to execute a simulation and verify the results using only the VTK Result Object as a source. That means that all solvers of FreeCAD are supported. If the solver doesn’t directely output a VTK Result one is automatically generated from the Result Object. Have a look at the following unittest module to get an idea of how such tests are written: https://github.com/ceeli/FreeCAD_bhb/blob/testing/src/Mod/Fem/femtest_new/test_problems.py
The library that enables that is the new fem test support library: https://github.com/ceeli/FreeCAD_bhb/blob/testing/src/Mod/Fem/femtest_new/support.py. Documentation will be added soon :wink:

Hi, the third coding phase is over and this is a summery of the current state of my testing project. This summery is divided into three parts: Python testing, C++ testing and source documentation (Python and C++).

The python changes are managed in a pull request on github. It contains two new modules to ease the development of FreeCAD tests. One for App tests (Mod/Test/AppTestSupport.py) and on for Gui tests (Mod/Test/GuiTestSupport.py). It also introduces a new way of managing collections of tests using the test packages directly (via the load_tests protocol). This is a little simpler in my opinion and avoids listing all tests again. There are now two subpackages femtest.app and femtest.gui containing tests that can be executed without the gui and those that require the gui respectively.

Test coverage of the python source can now be measured by using the new --cov option together with the -t option. This generates a report summery at the end of the test output and a html coverage report in the ./coverage directory. The html report shows which lines were executed for each file in detail. Optionally a package or module can be specified after the --cov option to limit the source files that are measured for coverage to the package or module. The test collections (e.g. femtest.app) specify which source files should be covered themselfs so no package or module must be specified to generate a useful report. See the documentation of src/App/FreeCADTest.py for more detail. The pull request also contains python source documentation and new test modules. A new very interesting kind of test can be found in femtest/app/test_problems.py. The tests in this module execute textbook examples and verify the results afterwards. The heavy lifting of these tests is done by the femtest/app/support_solver.py support module. All new files are documented so if you want to know more just look at the source :wink:.

I was less successful on the C++ side of things. There is a prototype which successfully tests the FemAnalysis and FemConstraint modules but I encountered a lot of problems when I tried to integrate the C++ unit tests into the test workflow of FreeCAD. My goal was to make it possible to execute C++ tests just like Python tests out of a running instance of FreeCAD but I could not produce a satisfying result by using googletest. The problem is that googletest doesn’t expose it’s internals. The only public interface is the command line interface which can be configured exactely once per program execution. This makes it impossible to do test case selection in a running instance of FreeCAD. I think we have to look at alternative unit test frameworks. I already had a look at boost.Test and Cpputest both of which expose a public API other than the command line interface. However I did create a pull request to incooperate the source documentation I created on github.

The last part concerns the source documentation and API documentation generation (html docs). I spent much more time on this part than I originally expected. I created three different prototypes:

Simpified C++ Doxygen
Simpified Python Doxygen
Sphinx Python

There are two related questions which need to be answered: Which tools to use and how to do source documentation? A discussion about this has started in this thread which will be continued in Consensus on API Documentation.

Why are you committing to Bernd’s repository? Is the idea that he reviews the stuff, and then a separate pull request is made against the main FreeCAD repository?

About the documentation, what exactly did you do? Did you just create the HTML templates, or did you set up an automated way to create the documentation in some clever way? As I said before, it would be great if you could write a report on how you used Sphinx. Then I could pick up that faster. I think we should really investigate the Sphinx+Breathe route which basically uses Doxygen for parsing C++ sources.

As for the current method of just using Doxygen for everything, I don’t dislike the documentation of Python, actually. It’s okay as long as the docstrings use a proper style, like numpy. See Python codeformating Draft, Arch in the regard of pep8 etc.

Now what I see more troublesome, is all those Python commands that are created by C++, or that are in XML files. I’m not sure how that all works. In some cases they seem to be documented inside the C++ sources, so they aren’t available to be parsed by Doxygen or Sphinx. Yorik has mentioned that maybe we should create some fake Python sources, just so they are picked up by Doxygen (or Sphinx). I think it would make sense to investigate this.

For example

class Module : public Py::ExtensionModule<Module>
{
public:
    Module() : Py::ExtensionModule<Module>("Drawing")
    {
        add_varargs_method("project",&Module::project,
            "[visiblyG0,visiblyG1,hiddenG0,hiddenG1] = project(TopoShape[,App.Vector Direction, string type])\n"
            " -- Project a shape and return the visible/invisible parts of it."
        );
        add_varargs_method("projectEx",&Module::projectEx,
            "[V,V1,VN,VO,VI,H,H1,HN,HO,HI] = projectEx(TopoShape[,App.Vector Direction, string type])\n"
            " -- Project a shape and return the all parts of it."
        );
        add_keyword_method("projectToSVG",&Module::projectToSVG,
            "string = projectToSVG(TopoShape[, App.Vector direction, string type, float tolerance, dict vStyle, dict v0Style, dict v1Style, dict hStyle, dict h0Style, dict h1Style])\n"
            " -- Project a shape and return the SVG representation as string."
        );

This all defines functions like Drawing.project(), Drawing.projectEx(), Drawing.projectToSVG(), and that’s it. That’s the entire documentation.

So a Python module with proper docstrings could be used to properly document the function.

def project(shape, direction=Vector(1,0,0), type="Unknown"):
    """Project a shape and return the visible/invisible parts of it.

    The long and extensive docstring would be here.

    Parameters
    -------------
    shape : some shape

    direction : Base::Vector3, optional

    type : str, optional
    """
    return Drawing.project(shape, direction, type)

In case of FEM only PRs this has been the work flow in recent years. Like a prereview.

How is the merge on this going?

@m42kus has yorik contacted you about wrapping things up?

No but this is a good reminder, I was on holiday directly after the GSoC but I’ll do some wrapping up now.

https://github.com/FreeCAD/FreeCAD/commits/88637ae

cd8f0e0

rebased and added pep8 code formating

https://github.com/berndhahnebach/FreeCAD_bhb/commits/gsoctestpy

the Spinx code is here:

https://github.com/berndhahnebach/FreeCAD_bhb/compare/6074ffa69a..10d0c6d25c
https://github.com/berndhahnebach/FreeCAD_bhb/commits/py_doc

any news on this?