The specification and the plan that looks very good. Quite clearly its a huge topic and to start with one idea is better than remain at the discussion stage forever, tough it might be wise to see it as a prototyping to learn what works good and what new ideas need to be incorporated.
What could be added to the specification would be use cases. I like what @yorik said about the user being the most difficult part. I am sure FreeCAD users will come up with a million different ways to manage their parts with that tool, e.g. in their favourite structure.
I think the user community could help the project by agreeing on a handful of key use cases than can be put to test with the earliest of implementations. I give an example.
User A designs a CNC project and needs a linear rail system. So he wants to look at available models with <19mm rail width.
User B designs freaky housing for an PCB project and want to look through the available potentiometers and buttons. He wants to see all parts for 6mm axis and potentiometers with threads.
User D wants to download all FreeCAD internet library objects from the “professional line” of the manufacturer “UK Doorgrips & Co”
If it is desired to run a remote sql server, this is likely not a significant issue. FreeCAD already has a web server somewhere. That host probably will have a LAMP stack or similar so it already has, or can easily have, a mysql server. No big deal to fire it up.
Who has the keys to the hosting? My guess would be yorik?
What are your ideas on tags. How should we finalize the tags? chennes yorik
I have an Idea, we can use the directory names of the Component Library for tags.
I have made a small script to do that:-
import os
tags = []
base_path = "<path>/FreeCAD-library"
[attachment=0]tags.txt[/attachment]
def get_directories(path: str):
if os.path.isfile(path):
return
tag = path.rsplit('/', 1)[-1]
tags.append(tag)
for p in os.listdir(path):
get_directories(os.path.join(path, p))
get_directories(base_path)
tags.remove("FreeCAD-library")
with open("test/tags.txt", "w") as file:
for tag in tags:
file.write(tag+'\n')
For populating the database, most of the fields can be extracted from the current Components Repository.
Metadata Fields :-
Required fields :-
name
version
maintainer
Optional fields :-
author
thumbnail
description
rating
Field SIZE will be the part of component data as it depends on the type of file (step, fcstd, etc)
The data I am able to extract :-
name
size
thumbnail
I think for the components present in the Repository, a default Maintainer may be given? and same for the Author? Description and Rating can be kept empty for now.
But what about the Version? I am note able to extract that from the Repository.
Any ideas? chennes yorik catman
Version is complicated. Depending on the base system, there could be several ways to get it. I would leave a field there but not populate it for now. Maybe at some point it’s a property we could have in a FCStd file… But then other filetypes might not have it. Maybe that’s something that should be scanned from the file’s timestamp and kept in a history table? Not sure…
There could be assets of different filetypes too (FCStd, Step, etc…). Maybe it’s an interesting field to have too?
I really like the tags. Is three any way to get a list of all available tags. In the video you show how to get the enter completion, but there you need to know what tags are available.
When the tags of a model are visible in list windows a context menu could allow them to be added to the filter list. That spares typing the name of the tags.
I think the prototype should get two major structural additions:
a switchable panel on the left side with a tree.
The tree is showing the structure of all items. When I click on a leaf, the list and all searches are limited to that leaf. There must be at least one underlying basic structure of the whole library. Default could be the filesystem structure.
The main window should get a switch between list view and your current view.
For the 200 you currently have it just takes too long to page though all of them when you have some 6-9 items per page.
A production version should have a list with customizable fields and it can be sorted by each field ascending/descending or filtered by a rule.
Another useful tag would be “manufacturer”.
I do not think fields should be made “required”. When I keep my onwn models in my local library. I do not want to be forced to enter “maintainer” each time. If nothing is edited the name could be extracted from the filename, for example.
A production version should have “Bulk renaming” for tags. Thats extremely efficient to use and so one could update large numbers of existing library files with usable tags. But the UI is quite complex for that so I think its out of scope for the prototype.
There could be options to define how to handle tag and value comparions, i.e. is Uppercase/lowercase and “contains” vs. “exact match”
Does the tool it work offline?
Does it already do something when a model is selected, like download and install or adding to a local database copy?
As soon as its testable it would be helpful for the ReadMe.md to get some text explaining what it is, dependencies and how to install it.
Hello Guys. After the completion of the Google Summer of Code period is over, I am continuing the project again.
I and one of my friend Harshita_Saraswat together are going to work the Component Management System and the Client Application.
A brief overview of what we will be working on are:
Component Management System
Validations:
Identification of required validation functions
Analyzing the data end points and the data they handle.
Consider the various types of and the specific validation requirements for each data type.
This could include component names , dimensions ,geometric properties etc.
Required validation functions would be created in validation module
Defining validation function for each type of data or validation requirement.
For example, we may create validateComponentName , validateSize that checks if they meet required criteria.
ElasticSearch:
Exploring the use cases of elasticsearch in our backend
ElasticSearch can be employed to index and search through component data such as names, properties etc.
It can be used for enabling quick and flexible retrieval.
Making an Object Relational Mapping(ORM) package for elastic search
This package will facilitate the interaction between the system and elasticsearch , abstracting away the complexity of working directly with
elasticsearch api.
The ORM package can include functions to create and manage elasticsearch indices, perform indexing and searching operations, and manage data mappings.
Using ORM with the api
This may involve importing the package and utilizing its functions within the system scripts.
This may include Indexing Component Data, Searching For Component, Updating and Deleting components.
API Improvement:
Add Authentication
It ensures that only authorized users can access and modify the designs and components.
Add Authorization
It ensures that authenticated users have appropriate permissions to perform specific actions. For example, not all users should be allowed to modify critical design files.
Adding Versioning
It is essential in modeling to track changes made to designs and components over time.
Dynamic Component Attributes:
Dynamic component attributes enable parametrization of models so that certain properties can be adjusted or changed dynamically.
Since each component belongs to different component classes For example, electronic ,mechanical etc. and subclasses, components have different attributes which may not be common across other components.
Example, Attributes of a screw may be
Material
2. Size
3. Thread Width
4. Head Type , etc.
Attributes of LED will not be same
Color
2. Voltage
3. Power
4. Size , etc.
A dynamic attribute system will enable the user to add different attributes to different Component as their wish.
Benefits:
Component specifications would be more accurate.
Naming components would be more generic.
Searching for components would be easier.
Logging:
Logger is a valuable tool to track and record various events, errors, and activities that occur within modeling . Logging helps diagnose issues, monitor the status of operations, and gain insights into the behavior of scripts.
Client Application
TLS Initialization Failed Error:
TLS errors are popping up for each network requests
Identification and Causes of TLS Error
Solutions to TLS Errors
Pop Up Notifications:
Implementing pop-up notifications is essential to provide users with timely and meaningful feedback about various events, errors and successful operations.
It enhances the user experience and ensures that they are informed about the state of their design and any potential issues.
Notifications for the following events are required to be handled:
Any errors
Download Completion
Successful Uploads , etc.
Component Uploading:
An endpoint and the module for uploading of components to users own github and its management is already built in the Component Management System. The Client application is not able to use it right now. The UI components for it are already in place but the multipart data post request is not working.
Identify and Fix Multipart Data Error
Improve the Interface for it
Add Component Editing View
View for Dynamic Attributes:
As dynamic attributes are proposed in CMS, Its corresponding views are also required at the Client side.
Add appropriate components in the detailed component view.
Add required components for improving the searching capabilities.
SSL Config:
It ensures secure communication and data protection especially when integrating with external services like elasticsearch or when handling
sensitive data.
It ensures that the data exchange is encrypted, authorized and intercepted.
Logging:
Logger is a valuable tool to track and record various events, errors, and activities that occur within modeling . Logging helps diagnose issues, monitor the status of operations, and gain insights into the behavior of scripts.
I have discussed the above proposal with chennes. All of your feedback will be helpful.
I am implementing authentication and authorization for the Component Management System.
Requirements:-
basic user authentication
allow the component owners to edit components
admin controls
Available options that I am considering (kindly suggest other methods if applicable):-
JWT (JSON Web Tokens) with id & password
OAuth Providers
Points to consider:-
For uploading a component, client need to use their personal github or a default github will be there if user do not wish to use personal github.
uploading of component file will be done by the client once the metadata is created successfully by the API.
even if the client wishes not to use personal github for uploading of components, the Authorization to edit is should be provided. This can be done by mapping the components with the user_id.
I am leaning towards Github OAuth. Reasons:-
client already needs to interact with github for component uploading if they want to use personal github.
Github OAuth is easy to use.
Issues in implementing Github OAuth (or any other OAuth I think):-
It will use browser. We can either use User’s default local browser or QtWebengine.
I have no experience with QtWebengine and I think it will be more tough to implement and time consuming.
Using local browser has its own challenge, getting the auth-token back to the API for successful authentication. I have a solution designed for it which is illustrated below.
Screenshot_20240111_205518.png
Screenshot_20240111_205445.png
Screenshot_20240111_211935.png
Kindly give your views kkremitzki chennes
I did some research and have come up with the following flow:-
Screenshot_20240113_141656.png
This new method uses the QtWebView rather than using the User’s Local Browser.