Open Hub Scheduled Downtime

There are two periods of scheduled downtime for the Open Hub over the next week or so.

The first will start tomorrow, Friday, April 28, when Black Duck’s IT team starts a network update project in our production data center. There may be a series of service interruptions through this weekend. We do not expect that the Open Hub will be down for any long period of time should it be unavailable, although these service interruptions will be unpredictable.

The other service interruption will start on Friday, May 5 at 8 PM EDT. We will be taking the Open Hub offline at this time in order to perform a significant architecture upgrade. Unfortunately, this work cannot be done during the IT work window because the network upgrade will mean, by definition, that the network and some systems will be unavailable.

We’re really excited by this architecture upgrade, which we’re calling the FIS – OH Database Split (FODS) project.  This work is separating our back end that provides the Fetch, Import, and SLOC operations from the Analysis and UI operations. This is the culmination of a significant amount of work over more than a year as we took incremental steps to mitigate production risk, expand our capabilities and scalability, and improve our website performance. To do this, we need to pull the site offline to make a backup, restore it to two systems, and then run a series of scripts to update the schemas for the new two database architecture, then all the applications need to be deployed and verified before we bring the entire set of systems back online. We expect that all this work will take the majority of the weekend and the Open Hub will be unavailable throughout this work.

As always, thank you so very much for being a member of the Open Source Software community and a part of the Open Hub.

About Peter Degen-Portnoy

Mars-One Round 3 Candidate. Engineer on the Open Hub development team at Black Duck Software. Family man, athlete, inventor

,

  • Jeffrey Altman

    Peter, might I suggest that Black Duck post tweets when a maintenance window begins and ends. The contents of this post are unavailable when the “blog.openhub.net” service is down for maintenance.

    • Good idea, Jeffrey, thanks for the suggestion. This weekend was difficult because site availability was based entirely on the work performed by the IT team and the best information we had was that access would by spotty. But we are going to have another window next weekend in order to deploy the architecture upgrade and will be more proactive.

  • Peter, I can tell you I’m excited about this upcoming update! It would be great if this update helps OpenHub stay closer to what happens in the projects it tracks and take less troubleshooting time from you and your team!

    Keep up the good work.

    • Hi Jos!

      As are we! This update will separate the back end Fetch, Import, and SLOC (FIS)processing from the project analysis and website presentation. I’ll be writing about it soon.

      I’ve mentioned in a few places that when we first deployed our FISbot code, it was screamingly fast. Since then, we grew the FISbot into a larger service, and started addressing low-level issues. In a “death by a thousand paper cut” way, we seem to have introduced a number of small issues that have resulted in inefficient update scheduling.

      Last week, our IT department overhauled the production data center networking and upgraded critical infrastructure. Now we’re upgrading our back end processing ability. After this deployment, we will be focusing on returning the job scheduler to its former level of throughput. It shouldn’t be long before we have restored projects to being updated every 24 – 72 hours.