Anamaria Stoica

My Mozilla Blog

Posts Tagged ‘BuildRequest

Build Run Report

with one comment

One push to hg.mozilla.org triggers off the Build System to generate a certain number of Build Requests (depending on the branch). All these build requests make up a Build Run. In a previous post I have covered in more detail its flow through the Build System and the Build Run Life Cycle.

The present post will focus on the Buildapi Report on Build Runs.

URL & Parameters

The report can be accessed at the following URL:

<hostname>/reports/revision/<branch_name>/<revision>

, <branch_name> := the branch name (e.g. mozilla-central, try, ...)
, <revision> := the revision number (first 12 characters)

Report Contents

1. Summary

The report starts by displaying some general information on the Build Run:

Build Run - Summary
Fields explained:

  • Revision – the revision number (first 12 chars)
  • No. build requests – number of Build Requests
  • e2e Time:
    • Duration – the End to End Time (Duration := Greatest Finish Time – Least When Timestamp), or how long it took for all Build Requests in this Build Run to complete
    • Least When Timestamp – the earliest timestamp of the Build Requests’ start times
    • Greatest Finish Time – the latest timestamp of the Build Requests’ finish times
  • Build Requests statuses break down (No. build requests := Complete + Running + Pending + Cancelled + Interrupted + Misc):
    • Complete – number of completed Build Requests
    • Running – number of still running Build Requests
    • Pending – number of still pending Build Requests
    • Cancelled – number of cancelled Build Requests
    • Interrupted – number of interrupted Build Requests
    • Misc – number of Build Requests having other statuses (should never happen)
  • Rebuilds – number of rebuilds
  • Forcebuilds – number of forced builds
  • Results – how many of the Build Requests were successful, with warnings, failed (and/or encountered exceptions) or other (usually still pending and running Build Requests)
  • Build Requests job type break down (No. build requests := Builds + Unittests + Talos):
    • Builds – number of builds
    • Unittests – number of unittests
    • Talos – number of talos

2. Details: Individual Build Requests

Next, the report presents information on the individual Build Requests making up the Build Run. If you are interested in how the Build Requests are fetched from the database and what the individual fields describing a Build Request mean, you might want to also read Build Requests Query.

The table displays a lot of information, and many of the parameters are internal and relevant only to how the Build System works.

Build Run - Build Requests Table

, continuing with:

Build Run - Build Request Table More Info

Demo

Build Run eae6bdacf6d2 Report Demo.

This is just a demo & works only for Build Run with revision number eae6bdacf6d2. All links outside the purpose of this demo were deliberately disabled. Enjoy!

Note: all table columns are sortable.

See Also: Average Time per Builder Report, End to End Times Report, Pushes Report, Wait Times Report

Written by Anamaria Stoica

November 12, 2010 at 10:28 am

Build Request Query

with 6 comments

Many of the reports (End to End Times Report, Build Run Report, TryChooser Report, Average Time per Builder Report, Builder Report) use BuildRequests as constructing blocks. In this post I will describe how BuildRequests are fetched from Buildbot’s scheduler database.

First of all, scheduler database has the following schema:

Scheduler Database Schema

Scheduler Database Schema

The information about one BuildRequest is spread among at least 5 tables: builds, buildrequests, buildsets, sourcestamps, sourcestamp_changes and changes. Which means there is no other way to fetch the data we need other than creating a big JOIN/OUTERJOIN for the 5 above mentioned tables. This rather unfriendly query is necessary as a result of scheduler database’s design to work optimal with Buildbot’s internal mechanisms rather than our current query’s need.

The actual query, using SQLAlchemy, looks like this:

b = meta.scheduler_db_meta.tables[‘builds’]
br = meta.scheduler_db_meta.tables[‘buildrequests’]
bs = meta.scheduler_db_meta.tables[‘buildsets’]
s = meta.scheduler_db_meta.tables[‘sourcestamps’]
sch = meta.scheduler_db_meta.tables[‘sourcestamp_changes’]
c = meta.scheduler_db_meta.tables[‘changes’]

q = outerjoin(br, b, b.c.brid==br.c.id) \
.join(bs, bs.c.id==br.c.buildsetid) \
.join(s, s.c.id==bs.c.sourcestampid) \
.outerjoin(sch, sch.c.sourcestampid==s.c.id) \
.outerjoin(c, c.c.changeid==sch.c.changeid) \
.select().with_only_columns([…]) \
.group_by(br.c.id, b.c.id)

Query explained:

JOINS:

  1. OUTERJOIN (LEFT OUTER JOIN) between buildrequests and builds tables – OUTERJOIN is required because some of the BuildRequests might be PENDING or be CANCELLED (thus having no builds, i.e. entries in the builds table)
  2. JOIN with buildests table on buildsetsid column (bs.id = br.buildsetid) – we need to go through the buildesets table in order to link the BuildRequests to the sourcestamps and changes information
  3. JOIN with sourcestamps table – sourcestamps information
  4. OUTERJOIN with sourcestamp_changes on sourcestampsid column (s.id = sch.sourcestampid) – linking further along to changes. An OUTERJOIN was necessary instead of an INNER JOIN, because the nightly builds don’t have a revision number or any entries in the changes table
  5. OUTERJOIN with changes table on changeschangeid column (sch.changeid = c.changeid) – OUTERJOIN again needed in order to include all BuildRequests belonging to nightly builds (see JOIN 4. above)

GROUP BY:
A final group by buildrequests.id and builds.id columns (GROUP BY br.id, b.id) is needed to capture multiple builds for the same BuildRequest. One BuildRequest might have multiple builds (usually very few and at most 2 or 3), if the builds have been retriggered or forced build manually.

Selected table columns explained:

  • b.number
  • b.c.start_time
  • b.c.finish_time
  • br.c.id.label(‘brid’)
  • br.c.buildername
  • br.c.submitted_at
  • br.c.claimed_at
  • br.c.claimed_by_name
  • br.c.complete
  • br.c.complete_at
  • br.c.results
  • br.c.buildsetid
  • bs.c.reason
  • s.c.id.label(‘ssid’)
  • s.c.branch
  • s.c.revision
  • c.c.when_timestamp
  • c.c.author
  • c.c.comments
  • c.c.revlink
  • c.c.category
  • c.c.repository
  • c.c.project

BuildRequest statuses:

  • PENDING – the BuildRequest has not started yet / no Build Master claimed it yet:
NOT b.start_time AND NOT br.claimed_at AND NOT br.complete AND NOT br.complete_at AND NOT b.finish_time
  • RUNNING – the BuildRequest is running (a Build Master claimed the BuildRequest already), and has not finished yet:
b.start_time AND br.claimed_at AND NOT br.complete AND NOT br.complete_at AND NOT b.finish_time
  • COMPLETE – the BuildRequest was completed without any internal errors or external interruptions (i.e. not CANCELLED / INTERRUPTED):
b.start_time AND br.claimed_at AND br.complete AND br.complete_at AND b.finish_time
  • CANCELLED – the BuildRequest was cancelled (i.e. it never got to start):
NOT b.start_time AND NOT br.claimed_at AND br.complete AND br.complete_at AND NOT b.finish_time
  • INTERRUPTED – the build was interrupted (e.g. slave disconnected) and Buildbot retriggered the build:
b.start_time AND br.claimed_at AND br.complete AND br.complete_at AND NOT b.finish_time
  • MISC – should never happen

BuildRequest results (buildrequests.results):
This column specifies how the BuildRequest execusion went, if it is completed. Naturally, the PENDING and RUNNING ones will have NO_RESULT:

  • -1 (NULL) – NO_RESULT
  • 0 – SUCCESS
  • 1 – WARNINGS
  • 2 – FAILURE
  • 3 – SKIPPED
  • 4 – EXCEPTION
  • 5 – RETRY

BuildRequest reasons (buildsets.reason):
The reason of the build, might be the scheduler (normal case), the nightly sheduler, a rebuild or forced build:

  • scheduler
  • nightly, e.g. ‘The Nightly scheduler named ‘Linux x86-64 mozilla-central nightly’ triggered this build’
  • rebuild, e.g. ‘The web-page ‘rebuild’ button was pressed by ‘<unknown>’: redo for slave disconect (nthomas)’
  • force build, e.g. ‘The web-page ‘force build’ button was pressed by ‘jhford’: hg poller is busted’

BuildRequest wait time:
How much the BuildRequest waited from when the change was created or the time it was submitted (only for nightlies because they have no changes) until the build has started (was assigned to a free slave):

change_time := c.when_timestamp, if c.when_timestamp != NULL
:= br.submitted_at, otherwise
WAIT_TIME := b.start_time – change_time, if b.start_time != NULL AND change_time != NULL
:= 0, otherwise

BuildRequest duration:
How long from when the change was created  or the time it was submitted (only for nightlies because they have no changes) until the build was complete, whether if successful or not:

change_time := c.when_timestamp, if c.when_timestamp != NULL
:= br.submitted_at, otherwise
DURATION := br.complete_at – change_time, if br.complete_at != NULL AND change_time != NULL
:= 0, otherwise

BuildRequest run time:
The actual run time of the build:

RUN_TIME := DURATIONWAIT_TIME

See Also: Wait Times Query, Pushes Query

Written by Anamaria Stoica

October 4, 2010 at 10:06 pm