OSUOSL datacenter migration and hosting costs for buildbot workers

[Etienne Champetier]

Hello Baptiste, Daniel,

Le lun. 7 juil. 2025 à 17:36, Daniel Pono Takamori
<pono at sfconservancy.org> a écrit :
>
> Heya, Pono from Software Freedom Conservancy (OpenWrt's fiscal home)
> here.
>
> Just wanted to quickly chime in a mention that we at SFC are working
> with the OSL, together with our member projects, to figure out the best
> way forward for hosted infrastructure. Sounds like it's been working out
> with hosting buildbot workers over there, so if that seems to be the
> preferred option, we can work on getting a contract with OSL (and find
> out hosting costs). On that note I wanted to mention that there are
> funds available at SFC for things like this, so we can avoid having
> members pay for CI!
>
> I'll update again after we get a chance to talk internally and with
> Lance at OSUOSL about the best way forward for SFC and its member
> projects to continue to host over there.
>
> Thanks!
> -Pono
> Community Manager
> Software Freedom Conservancy
>
> On Mon, Jul 07, 2025 at 11:21:53PM +0200, Baptiste Jonglez wrote:
> > Hi,
> >
> > OSUOSL is migrating to a new datacenter that will increase their hosting
> > costs, and they want to discuss how we could possibly contribute to costs.
> > Details quoted below.
> >
> > They currently provide us 6 build workers (2 physical hosts and 4 VMs):
> >
> >   https://openwrt.org/infrastructure#buildbot_workers
> >
> > This infra has been working quite well since 2018, with different
> > iterations.  We haven't paid anything so far.
> >
> > I see several possible directions:
> >
> > 1) establish a contract and start paying OSUOSL for the service using OpenWrt funds
> >
> > 2) stop using these buildbot workers
> >
> > 3) keep using these buildbot workers for free (same as today)
> >
> > Personally I would prefer to continue working with OSUOSL, and I think we
> > have the funds to contribute financially.  They have been very
> > professional and responsive, the level of service is the same or even
> > better than companies like Hetzner.  We don't need to spend too much time
> > since they take care of everything (e.g. initial OS installation, hardware
> > repairs)
> >
> > We could even maybe increase the number of workers: it seems more sustainable
> > than having project members paying buildbot workers from their own pockets.
> >
> > What do you think?

If you can share some more details, ie breakdown of expected price per
months (power / bandwidth) and what servers we are running than would
help
If the hardware is from 2018, it might be cheaper to rent newer
hardware at one of the cheap hoster (OVH / hetzner) than continue to
run old servers


> > OSUOSL request below:
> >
> > > Due to significant infrastructure issues at the Kerr Datacenter, including a
> > > critical failure in one of the two main UPS systems and the high cost of
> > > necessary repairs, Oregon State University has made the decision to vacate this
> > > facility rather than invest further in it. As a result, OSL is required to
> > > relocate all of its equipment within the next few months (potentially sooner).
> > >
> > > Relocating to a new datacenter will increase OSL's operational costs for power,
> > > space, and connectivity compared to Kerr. Historically, OSL covered most costs,
> > > but higher expenses in new facilities mean sustainable hosting for dedicated
> > > physical hardware requires projects to contribute more directly to covering
> > > these operational expenses.
> > >
> > > For projects with existing contracts:
> > >
> > > We don't anticipate an immediate increase in your contractual costs. However,
> > > higher OSL costs overall may necessitate future discussions about pricing upon
> > > contract review/renewal.
> > >
> > > For projects without contracts:
> > >
> > > Facing significantly higher operational expenses in a new facility compared to
> > > Kerr, sustainable hosting for dedicated physical hardware will require projects
> > > to contribute more directly to covering these operational expenses. We
> > > understand that fully covering the total costs might be challenging for some
> > > projects, and we are flexible in working with you to find a solution. Our aim is
> > > to ensure the costs directly associated with hosting your physical equipment are
> > > addressed to make continued co-location sustainable.
> > >
> > > We want to partner with you to find a workable solution for contributing to
> > > these costs. We offer two primary paths for contribution: establishing a formal
> > > hosting contract (our preferred method) or providing an annual donation based on
> > > an estimate of your hosting costs. We are committed to working with you to find
> > > a mutually agreeable solution and will not remove any co-located hardware until
> > > a solution is in place. Please let us know in your reply if you would like to
> > > discuss these contribution options in more detail.
> >