Anamaria Stoica

The Wait Times Query is very similar to Build Request Query, only that it fetches jobs (does not care about multiple builds of the same BuildRequest), it selects a different subset of columns and has several other restrictions in addition.

The base query, using SQLAlchemy, looks like this:

q = outerjoin(br, b, b.c.brid == br.c.id) \
.join(bs, bs.c.id == br.c.buildsetid) \
.join(s, s.c.id == bs.c.sourcestampid) \
.outerjoin(sch, sch.c.sourcestampid == s.c.id) \
.outerjoin(c, c.c.changeid == sch.c.changeid) \
.select().with_only_columns([...])
# multiple restrictions
.group_by(br.c.id)

For the meaning of the JOINs and the tables involved, see Build Request Query. In this post, I’ll continue by describing only the differences (placed where commented #more restrictions):

1. Pool selection – fetching the jobs belonging only to a pool

This is done by filtering jobs claimed by masters in the selected pool (i.e. by looking at values of buildrequests.claymed_by_name column). There are currently 3 pools: ‘buildpool’, ‘trybuildpool’ and ‘testpool’, each having a different number of masters. For example, buildpool has 4 masters:

• ‘production-master01.build.mozilla.org’
• ‘production-master03.build.mozilla.org’
• ‘buildbot-master1.build.scl1.mozilla.com:/builds/buildbot/build_master3′
• ‘buildbot-master2.build.scl1.mozilla.com:/builds/buildbot/build_master4′

The masters in each pool are specified by BUILDPOOL_MASTERS in buildapi.model.util module.

One exception are PENDING jobs, as they haven’t been claimed by any master yet (buildrequest.claimed_by_name is NULL). However, it is possible to tell which pool they belong to by looking at buildrequests.buildername‘s value:

• buildpool: br.claimed_by_name is NULL AND br.complete = 0 AND br.buildername NOT LIKE ‘Rev3%’ AND br.buildername NOT LIKE ‘% tryserver %’
• trybuildpool: br.claimed_by_name is NULL AND br.complete = 0 AND br.buildername NOT LIKE ‘Rev3%’ AND br.buildername LIKE ‘% tryserver %’
• testpool: br.claimed_by_name is NULL AND br.complete = 0 AND br.buildername LIKE ‘Rev3%’

(where br is buildrequests table)

2. Timeframe filtering

Filters out only the jobs with the change’s timestamp in the interval [starttime, endtime). The change’s timestamp is specified by changes.when_timestamp column, except for the nightly builds that have no changes. In those cases we’ll look at buildrequest.submitted_at values (which are usually at most a few minutes later).

q = q.where(or_(c.c.when_timestamp >= starttime, br.c.submitted_at >= starttime))
q = q.where(or_(c.c.when_timestamp < endtime, br.c.submitted_at < endtime))

3. Rebuilds and forced builds exclusion

All rebuilds and forced builds are excluded from the stats. This is done by looking at buildsets.reason column, and filtering out values found in buildapi.model.util.WAITTIMES_BUILDSET_REASON_SQL_EXCLUDE.

4. Exclude buildernames that are not of interest, like fuzzers

The exclusion list is specified by buildapi.model.util.WAITTIMES_BUILDREQUESTS_BUILDERNAME_SQL_EXCLUDE.

See Also: Build Request Query, Pushes Query, Wait Times Report.