It looks like you're new here. If you want to get involved, click one of these buttons!
GitHub will discontinue Subversion support, which is the basis of all the packages hosted in KLayout (https://github.blog/2023-01-20-sunsetting-subversion-support/). KLayout uses the SVN interface to download the package data.
As a quick mitigation, it is possible to host packages on SourceForge with Subversion selected as the code tool. This will mean manual effort upon releasing GitHub-hosted projects. This used to be very convenient with the subversion bridge.
To facilitate this migration I will add SourceForge to the package repositories and remove GitHub.
Existing packages will be transferred to a read-only KLayout-owned SVN repository so they will continue working.
As a long-term solution, I plan to add Git support for package download. That means:
grain.xml
needs to be located at top level). So far, it was possible to use a sub-folder as the package home. This will no longer work. A workaround is to use a Git submodule for the package part.There is a ticket for this: https://github.com/KLayout/klayout/issues/1514
Matthias
Comments
Here is a brief recipe for the transfer to SoureForge. The example is for the klayoutmatthias/xsection project.
First, create a project at SourceForge. Make sure you select "Subversion" for the Repository type. In my case the project was "klayoutmatthias-xsection". You should be able to navigate to the project and on the "Code" page and find SVN checkout instructions. In my case there are two ways:
SSH: via URL
svn+ssh://koefferl@svn.code.sf.net/p/klayoutmatthias-xsection/code/
HTTPS: via URL
https://svn.code.sf.net/p/klayoutmatthias-xsection/code/
We will use SSH for the import and HTTPS as the deployment path. For SSH you need to register your public SSH key in your User Profile.
I want to import tags only for the releases, so I created a "tags" folder in the Subversion repo:
To transfer a GitHub release (tag) to a Subversion tag on SourceForge, I use these commands for the 1.7 tag:
With the SourceForge mirror in place, I can substitute the original GitHub URL by the SourceForge URL:
becomes
Note, that the package data itself is not located directly in the top level of the module, but inside the "src" subfolder. Hence, we need to add "src" to the tag path.
I have started duplicating the first project tags to a separate SVN server. This will keep the packages working after January 2024. But as these are snapshots take today, this will disconnect them from the evolution on GitHub.
The new package URLs will be formed by replacing
https://github.com
withhttps://www.klayout.org/svn-public/packages-migrated
.Technically, the SVN repo is a snapshot of selected pieces from the GitHub space.
Matthias
I find GitHub baffling and don't think I've ever found
what I'm looking for, there. I always took the latest off
either SourceForge or the klayout site "Get" links. As
long as those have good latest / greatest, GitHub can
disappear for all I care.
Hi Matthias,
I see that SourceForge provides an automated synchronization between GitHub and SourceForge. Could that be useful?
https://sourceforge.net/p/import_project/github/
When do you anticipate having v0.29 and a new Git integration released? If before Jan 2024, then perhaps we can wait for that instead of using the SVN workaround?
In your top-level approach, would there be mechanism to only include specific subfolders, or to exclude specific folders?
thank you
Lukas
Hi Lukas,
I tried the sync feature of SourceForge, but as I understand it, it can only synchronize Git to Git and not Git to SVN.
I am currently implementing git support for download. git cannot efficiently download subtrees - it will always download the worktree for the full repository and that makes retrieving a single file like grain.xml rather inefficient. There is shallow and sparse checkout which helps mitigating the effects somewhat. Still things will be considerably slower with git.
Maybe it is possible to support sub-folders by pruning everything above. However, it still is required to download the whole tree first. In case of https://github.com/siepic/SiEPIC_EBeam_PDK.git for example that means downloading 277MB of data, just to retrieve grain.xml.
Matthias
If it is a question of accessing all the grain.xml files, what about just queering GitHub for the single file? e.g.,
https://raw.githubusercontent.com/SiEPIC/SiEPIC_EBeam_PDK/master/klayout/grain.xml
Putting the grain.xml file at the top of the repo should be ok.
Hi Lukas,
It's unfortunately the Git protocol is very bad in delivering single files only. But I found a solution that is acceptable. It involves caching all the repository description files on the Salt Mine server and using that cached information inside KLayout. So there is little need to download the full package except if someone installs it.
It took some rework of the server application plus I had to upgrade the Windows builds and now everything is breaking. But I am working on that. Basically a solution is at sight
Matthias