Depending on your platform different preparation is necessary to build locally:
For linux it’s best to install docker.
For windows you need a recent visual studio compiler.
For osx a macosx sdk is necessary.
Incremental builds for debugging is something more difficult as it is not really supported. Ideally conda adds support for this and ide’s get aware of environments.
You can setup everything manually but this is a lot of work and far from easy. If all you want is python-development you can simple replace the files you want to modify (eg.: via link)
After some research I found a tool that I think will help with getting started with incremental builds. It’s called “conda debug” [1] and is very similar to “conda build” but only gathers all the dependencies and surces. Building has to be done manually by calling “bash conda_build.sh”
For linux (ubuntu) I had to modify the recipe a bit to work. Somehow there are still some system libraries used which are in /usr/lib64 for centos and on /usr/lib/x86… for ubuntu. So I simple created a link from the latter to the first to get rid of these cmake errors. The other change was regarding opengl library not found. Here a toolchain file for cmake helped to solve the problem [2].
But now it’s happily building and I guess doing a git pull in the work directory and afterwards again call “bash conda_build.sh” will only build the updated libraries.
So you see, support for incremental building is coming and c++ ide’s will hopefully accept the importance of virtual envs.
....
source tree in: /Users/bbl/miniconda/conda-bld/debug_1570470681145/work
################################################################################
Test environment created for debugging. To enter a debugging environment:
cd /Users/bbl/miniconda/conda-bld/debug_1570470681145/work && source /Users/bbl/miniconda/conda-bld/debug_1570470681145/work/build_env_setup.sh
To run your tests, you might want to start with running the conda_test_runner.sh file.
################################################################################
Nice, I think osx should work similar to Linux. I will try the same on Windows. Would be nice to have a cross platform way to setup a development environment for freecad.
ninja: error: ‘/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/AppKit.framework’, needed by ‘Mod/Import/Import.so’, missing and no known rule to make it
rm: /Users/bblockmans/miniconda/conda-bld/debug_1570470681145/_h_env/share/doc/FreeCAD: No such file or directory
since Mod/Import/Import.so is refering tho the file
but this seams OK to me since the LC_PATH exists and the buils could apparently go on?
finaly , just like in your vimeo, I had to add 2 missing modules as highlighted by launching FreeCAD
During initialization the error “No module named '> numpy> '” occurred in /Users/bbl/Library/Preferences/FreeCAD/Mod/A2plus/InitGui.py
Please look into the log file for further information
During initialization the error “No module named ‘six’” occurred in /Users/bbl/Library/Preferences/FreeCAD/Mod/flamingo/InitGui.py
Please look into the log file for further information
No module named '> six> ’
after that I could launch FreeCAD and succesfully test some of its features.
Just have a systematic “segmentation fault” error
objc[34939]: Class FIFinderSyncExtensionHost is implemented in both /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit (0x7fffa0db13d8) and /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride (0x131748f50). One of the two will be used. Which one is undefined.
Segmentation fault: 11
when opening one of my basic test file, but I’ll check on the conda “daily build” to see if I can reproduce this last error.
Many thanks for the explainations.
for a better understanding off this all, I’ll have to play and test it a bit further. But it looks really cool and promising.
Looo you probably have an idea on what’s explained below?
in the work dir of my build env I can’t see the latest git commit (i.e. those from today).
Indeed git log on master branch in the /work dir of my conda debug/build env shows the latest commit dating from the 7th of october, while on Freecad/Freecad github the latest commits are from today the 8th of october?
What I understand is that by running the
conda debug . -m .ci_support/osx_python3.7.yaml
a git_cache has been created in my debug/build conda env but in the state the github repository was at that time. This cache is then used as my remote/origine repository for my conda debug/build env (cf. the work dir). Hence I am not aware of more recent commits on github.
What would be the recommended way for updating the my debug/build conda env git_cache? i.e. how tho resync my cache with the actual state of the github repo?
Can I do it with a git command
should I use a conda command like “conda update …”
do we have to rebuild the all debug env and then rebuild freecad (which solution would be quite time consuming)
What would be the recommended way of pulling these recent commits in my conda debug env since git status gives to folowing output i.e. it looks that I am
Also note that some extra files are added in the directory, so be careful with “git add” to not include these files in a PR…
It seems like there are always some manual steps necessary to setup the development environment…
What would be the recommended way of pulling these recent commits in my conda debug env since git status gives to folowing output i.e. it looks that I am
I guess there is no recommendation. This is quite experimental.
OK maybe it’s best to consider our conda debug/build git repo (i.e. the one that is in the git-cached folder) as a kind of “fork” of the original and try to keep it in sync with the github/Freecad original repo.
The idee is that we can keep the master branch of our “FORK” repo in sync with the upstream master branch (fetch and pull/commit from github on the fork). It means never change anything on the master branch except from the upstream sync, And make our development on a dedicated new branch in our “Fork” repo.
maybe it’s even easier to propose a pull request at the end of a developpment cycle?
I guess the inconvenience we are seeing here are due to feedstocks are not really designed for the creation of a development environment. Also conda debug is not meant to address this purpose. I guess this tool should provide maintainers the ability to run a recipe several times without going through the whole process of gathering source and dependencies.
So some inconvenience are expected, but in my eyes “conda debug” is pretty close to the things we want to achieve here.
There are several options how to solve this. One might be to specify source in another directory which can be done by commenting out the source section in the meta.yaml and set the directory in the build script. We could add these changes to a fork/branch of the freecad-feedstock, but I learned in the past, having different variants of the repo will increase maintenance work and pretty soon we see things getting out of date and so on.
So in my eyes, for now, the simplest way to solve this is switching the remote to point to whatever branch you want to follow.
moving this discussion from occt7.4 topic to this topic.
Today I tested conda-debug on windows and it seems to work the same way as it does on linux. So if you have visual studio installed (1) you can follow pretty much the same instructions as shown for linux in this video [1]. This is:
install miniconda (if you haven’t done already, this is not done in the video)
conda install conda-build (if you haven’t done yet)
This will gather all the things necesarry to build freecad (deps, sources via git) and will need quite some time (depending on your internet-connection). Once done there will be a output on how to proceed (changing into the work directory and activating the build-env on top of the host-env). As @danreb has figured out there is one additional step necessary to pull sources from the freecad-repo once the currently used sources are out of date. So the following:
(If you are used to git, cmake and know how it works you can also do any other workaround, for example you can modify the conda_build.bat file to look for the yource in another directory)
Once this is done you simple call the build script:
conda_build.bat
and wait until the build has finished. Once done you can start freecad by this command:
FreeCAD
Now you can create a script which does all the stuff at once, eg. this is how it looks for me:
cd C:\Users\lo\Miniconda3\conda-bld\debug_1571046021538\work
call C:\Users\lo\Miniconda3\conda-bld\debug_1571046021538\work\build_env_setup.bat
git pull
call conda_build.bat
FreeCAD
Now you simple have to doubleklick this script and latest freecad gets pulled and compiled. I hope this helps.
(1) I guess 2015 / 2017 / 2019 should work. Microsoft changes it’s strategy pretty fast regarding the compilers and the download page also changes quite often, so I can’t make a 1., 2., 3. suggestion on how to install visual studio compilers. In the long run I hope conda packages for windows will switch toolchain to a more openly distributed set of compilers (clang, flang, …)
edit: not yet sure if the script approach really works. Somehow I see a crash during linking but I am not sure what the reason for this is.
edit1: currently we use a patch for pyside which is necessary for windows. This patch must be applied otherwise the build is abborted. This was the reson for the aborted build which I mentioned in the first edit. Therefor I applied the script and switch to another bbranch and use rebase instead off pull to get the latest source.
(base) D:\FreeCAD-feedstock>conda debug . -m .ci_support\win_cxx_compilervs2015p
ython3.7.yaml
No numpy version specified in conda_build_config.yaml. Falling back to default
numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.ya
ml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Adding in variants from D:\FreeCAD-feedstock\recipe\conda_build_config.yaml
INFO:conda_build.variants:Adding in variants from D:\FreeCAD-feedstock\recipe\co
nda_build_config.yaml
Adding in variants from .ci_support\win_cxx_compilervs2015python3.7.yaml
INFO:conda_build.variants:Adding in variants from .ci_support\win_cxx_compilervs
2015python3.7.yaml
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed
Traceback (most recent call last):
File "C:\Miniconda\lib\site-packages\conda_build\environ.py", line 757, in get
_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "C:\Miniconda\lib\site-packages\conda\common\io.py", line 88, in decorate
d
return f(*args, **kwds)
File "C:\Miniconda\lib\site-packages\conda\plan.py", line 474, in install_acti
ons
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "C:\Miniconda\lib\site-packages\conda\core\solve.py", line 117, in solve_
for_transaction
should_retry_solve)
File "C:\Miniconda\lib\site-packages\conda\core\solve.py", line 158, in solve_
for_diff
force_remove, should_retry_solve)
File "C:\Miniconda\lib\site-packages\conda\core\solve.py", line 275, in solve_
final_state
ssc = self._add_specs(ssc)
File "C:\Miniconda\lib\site-packages\conda\core\solve.py", line 555, in _add_s
pecs
explicit_pool = ssc.r._get_package_pool(self.specs_to_add)
File "C:\Miniconda\lib\site-packages\conda\resolve.py", line 523, in _get_pack
age_pool
pool = self.get_reduced_index(specs)
File "C:\Miniconda\lib\site-packages\conda\common\io.py", line 88, in decorate
d
return f(*args, **kwds)
File "C:\Miniconda\lib\site-packages\conda\resolve.py", line 544, in get_reduc
ed_index
explicit_specs, features = self.verify_specs(explicit_specs)
File "C:\Miniconda\lib\site-packages\conda\resolve.py", line 280, in verify_sp
ecs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound:
- qt=5.12
- doxygen
- netgen
- coin3d
- hdf5=1.10.5
- boost-cpp=1.70.0
- blas=[build=openblas]
- occt=7.3
- smesh
- pivy
- pyside2
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Miniconda\Scripts\conda-debug-script.py", line 10, in <module>
sys.exit(main())
File "C:\Miniconda\lib\site-packages\conda_build\cli\main_debug.py", line 94,
in main
return execute(sys.argv[1:])
File "C:\Miniconda\lib\site-packages\conda_build\cli\main_debug.py", line 70,
in execute
activation_string = api.debug(thing_to_debug, verbose=(not _args.activate_st
ring_only), **_args.__dict__)
File "C:\Miniconda\lib\site-packages\conda_build\api.py", line 451, in debug
metadata_tuples = render(recipe_or_package_path_or_metadata_tuples, config=c
onfig, **kwargs)
File "C:\Miniconda\lib\site-packages\conda_build\api.py", line 45, in render
permit_unsatisfiable_variants=permit_unsatisfiable_variants)
File "C:\Miniconda\lib\site-packages\conda_build\render.py", line 819, in rend
er_recipe
allow_no_other_outputs=True, bypass_env_check=bypass_env_check)
File "C:\Miniconda\lib\site-packages\conda_build\render.py", line 725, in dist
ribute_variants
bypass_env_check=bypass_env_check)
File "C:\Miniconda\lib\site-packages\conda_build\metadata.py", line 1018, in p
arse_until_resolved
bypass_env_check=bypass_env_check)
File "C:\Miniconda\lib\site-packages\conda_build\metadata.py", line 940, in pa
rse_again
bypass_env_check=bypass_env_check),
File "C:\Miniconda\lib\site-packages\conda_build\metadata.py", line 1528, in _
get_contents
rendered = template.render(environment=env)
File "C:\Miniconda\lib\site-packages\jinja2\asyncsupport.py", line 76, in rend
er
return original_render(self, *args, **kwargs)
File "C:\Miniconda\lib\site-packages\jinja2\environment.py", line 1008, in ren
der
return self.environment.handle_exception(exc_info, True)
File "C:\Miniconda\lib\site-packages\jinja2\environment.py", line 780, in hand
le_exception
reraise(exc_type, exc_value, tb)
File "C:\Miniconda\lib\site-packages\jinja2\_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "D:\FreeCAD-feedstock\recipe\meta.yaml", line 43, in top-level template c
ode
- eigen
File "C:\Miniconda\lib\site-packages\conda_build\jinja_context.py", line 230,
in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "C:\Miniconda\lib\site-packages\conda_build\render.py", line 154, in get_
env_dependencies
channel_urls=tuple(m.config.channel_urls))
File "C:\Miniconda\lib\site-packages\conda_build\environ.py", line 759, in get
_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies
for platform win-64: {'occt=7.3', 'boost-cpp=1.70.0', 'smesh', 'blas=[build=open
blas]', 'pyside2', 'hdf5=1.10.5', 'netgen', 'qt=5.12', 'coin3d', 'pivy', 'doxyge
n'}
I could install miniconda successfully in advance. I have MSVC 2017 installed and can successfully use it to build FC using the LibPack.
