Switch issues and CI to GitHub

Tue Jan 18 06:38:43 PST 2022

Hi all,

Thanks for the active discussion. My thoughts on the three topics bug tracker, CI and Git _root_.

## Bug Tracker

I looked today into migrating issues from bugs.openwrt.org over to GitHub.com, codeberg.org (GiTea) and todo.sr.ht (Sourcehut). The migration path is somewhat easy extending the fly-build-csv[1] to export tickets and comments from a Flyspray SQL dump.

The resulting tasks.csv and comments.csv can be exported to the bug tracker of choice.  While sr.ht allows to import the large collection of issues, each message is limited to about 16.000 characters which would require us to truncate existing tasks and comments (and instead have them on some paste service). This limit is likely tied to the first class email support, users without an account can write to a special email address and create tickets without registering at all. Try it by sending something to ~aparcar/openwrt-bugs-import-test-2 at todo.sr.ht

Apart from that there (might be) are minor bugs which is totally understandable since sr.ht is still considered Alpha[2]. I’d prefer to give it more time before considering moving over there. If we decide to move there, tools like gh2srht[3] would allow a quick migration. To get a feel what the bug tracker over at sr.ht would look like I migrated as much as possible, feel free to have a look[4].

Migrating existing issues to GitHub seems unfeasible, while it’s technically possible[5] their rate limits prevent me (or a bot account) to effectively transport open issues and comments. In case we choose GitHub, a static version of bugs.openwrt.org should stay online.

Lastly I looked at codeberg.org which runs Gitea(.com) under the hood. From my understanding it’s a German _registered association_ (eingetragener Verein) with the goal to provide code hosting - great. Their API is well documented[5] and in no time at all I migrated some 4000 issues including comments[6]. It looks and feels like GitHub with some extra buttons, like you can assign issues to specific branches!

A quick bug tracker conclusion, I’d be happy to use codeberg.org for issue tracking. Both sr.ht and codeberg.org are FOSS, GitHub not so much. It's possible to migrate all existing issues to codeberg and turn off bugs.openwrt.org entirely, no need to host a static copy. They also allow to login via GitLab.com and GitHub.com accounts lowering the bar for user to participate.

If this seems a feasible option I’d rework the migration script and create them with a bot account, including username and time/date in the issues.

As an immediate action, we might as well close down bugs.openwrt.org and open issues on GitHub.com without any migration of existing issues. Both users and developers already know the workflows over there and issues have a higher visibility. A migration away from GitHub over to coderberg or sr.ht is possible with much less effort than migrating away from flyspray.

## CI

Codeberg does not offer any CI at all except an integration for a self hosted Drone CI setup[7]. That setup would keep the burden of server maintenance on our side, which makes it unfeasible.

On the other hand sr.ht offers excellent CI in many flavors[8] (except macOS) which even allows to login on runners to manually debug failed runs. On all machines it’s possible to install Docker and run our own containers or trust in the package stability of Debian et al. I tested it in the past and it works great[9].

GitHub offers free CI which is already heavily used from our side. The package.git repository ran about 11.000 times since I added it in September 2020. They offer up to 40 parallel running jobs which could even take over snapshot building (about 2 hours per target).

A quick CI conclusion, I’d start using GitHub CI for openwrt.git since we’re burning plenty of CPU and I don’t want to annoy sr.ht users and maintainers by flooding their infrastructure. In practice that would mean we have a folder called .github/ in our repository. However, we could also use some of the donated money and donate it ourselves to sr.ht hoping they invest in more CI runners.

## Git _root_

We’re currently maintaining multiple servers to host our many Git repositories (GitHub is just a mirror). Instead we could move to one of the three services and use them as our Git _root_.

From what I know sr.ht does not offer any organization accounts for now, we’d have to use a shared one until it’s implemented. Codeberg and GitHub do support organizations.

Sourcehut devs are the only ones thinking about supporting pre-commit hooks to allow "Signed-of-by” checks and more, something we have on git.openwrt.org. A workaround for Codeberg and Github is to disable direct pushes and only ever allow merging of PRs, which is a bit of a sad workflow downgrade.

All three of them support 2FA, Codeberg support U2F and soon FIDO2[10], GitHub support FIDO2.

All three play fine with in issue references in commit messages, meaning we can have `fixes: #42` or similar[11] in the commit message and issues are automatically closed.

To have an idea of the interface I cloned the openwrt.git repository over to sr.ht[12] and Codeberg[13].

A quick conclusion of Git _root_, I’d move git.openwrt.org and all archives over to Codeberg and disable merge requests there. Keep merge requests (and CI) on GitHub. Everyone with commit access should (must) use pre-commit[14] and run the bare minimum checks like “Signed-of-by” locally.

## Bonus

All three services support CLIs managing most common stuff, allowing to never touch the web interface; `hut` for sr.ht[15], `tea` for Codeberg[16] and `gh` for GitHub[17].

## Conclusion

From a FOSS perspective I’d skip GitHub entirely and move to Codeberg or sr.ht. Codeberg (Gitea) is a fine clone of GitHub and sr.ht comes with a great _no bloat_ attitude and priority on email integration for tickets and git (they created git-send-email.io).

On the other hand, all community repositories are on GitHub, people are actively and happy contributing there and mostly think about “how to make OpenWrt better” and less “how to improve our workflow and infrastructure”.

What do?

Paul

[1]: https://gist.github.com/sourceperl/ff726cfad9a083b9fe73a8479991f895
[2]: https://sourcehut.org/alpha-details/
[3]: https://sr.ht/~emersion/gh2srht/
[4]: https://todo.sr.ht/~aparcar/openwrt-bugs-import-test-2
[5]: https://codeberg.org/api/swagger
[6]: https://codeberg.org/aparcar/openwrt/issues
[7]: http://drone.io
[8]: https://man.sr.ht/builds.sr.ht/compatibility.md
[9]: https://builds.sr.ht/~aparcar/job/578105
[10]: https://github.com/go-gitea/gitea/pull/17957
[11]: https://man.sr.ht/git.sr.ht/#fixes
[12]: https://git.sr.ht/~aparcar/openwrt
[13]: https://codeberg.org/aparcar/openwrt
[14]: https://pre-commit.com
[15]: https://sr.ht/~emersion/hut/
[16]: https://gitea.com/gitea/tea
[17]: https://github.com/cli/cli

> On 7. Jan 2022, at 10:34, Paul Spooren <mail at aparcar.org> wrote:
> 
> Hi all,
> 
> Back at the Hamburg meeting in 2019 and a succeeding vote we decided to migrate over to a self-hosted GitLab instance. Some years passed and nothing really happened so I’d like to give this another go.
> 
> None of the OpenWrt project members is willing to setup and maintain a GitLab instance and there were multiple vetos again gitlab.com.
> 
> Our current bug tracker at bugs.openwrt.org is used by a minority of users (and devs), all community repositories (LuCI, packages, …) use GitHub for issue tracking. Instead of maintaining flyspray and the server, I’d like to export all flyspray issues, migrate them to GitHub and open GitHub issues for openwrt/openwrt to the public. A static or read-only version of flyspray could be hosted for the near future.
> 
> Secondly I’d like to give the CI of the main repository another go. Our CI to build Docker images is currently on gitlab.com, I’d migrate that over to GitHub. Also I’d suggest to add similar CI checks as added to the packages (and routing and video and LuCI) repository. We could compile targets and tooling, check checksums etc, even build snapshots to lower the resource usage of our Buildbot infrastructure.
> 
> During a recent _poll_ in #openwrt-adm multiple members liked the idea, however before doing or voting on anything, I’d like to ask for more comments.
> 
> Thanks for all feedback,
> Paul