Anamaria Stoica

My Mozilla Blog

Posts Tagged ‘Pushes

Introducing the Pushes Report

with one comment

The Pushes Report counts the number of pushes per branch within a selected timeframe. This number is equivalent with the number of pushes listed in Mercurial’s pushlog, like for example the mozilla-central pushlog.

In addition, the report brakes down the number of pushes per time intervals, listing for example how many pushes were there every 2 hours within a day, or every day within a month or week, or even every month (~30 days) within a year.

To see exactly how pushes are fetched from the Scheduler Database, and what restrictions are applied on them, see Pushes Query.

URL & Parameters

The Pushes Report can be accessed at the following URL:

<hostname>/reports/pushes?(<param>=<value>&)*

Parameters (all optional):

  • format – format of the output; allowed values: html, json, chart; default: html
  • starttime – start time, UNIX timestamp (in seconds); default: endtime minus 24 hours
  • endtime – end time, UNIX timestamp (in seconds); default: starttime plus 24 hours or current time (if starttime is not specified either
  • int_size – break down results per time intervals of this length if value > 0; values are specified in seconds; default: 7200 (2 hours)
  • branch – comma separated list of branches; displays results only for branches in the list; default: ” (all)
  • tqx – used by Google Visiualization API (automatically appended by the library), relevant only if format=chart; default:

Examples

1. Daily Report

  • timeframe: Sep 22 (1 day)
  • interval size: 2h (7200s)
  • branches: all (default)
  • url: <hostname>/reports/pushes?starttime=1285138800&endtime=1282546800

Notes:

  • busier in the middle of the work hours, less busy at night, however this is not the usual case! The number of pushes during a day varies very much from a day to another. The day of the week, release development cycle stage or other external factors greatly influence how the chart looks.

Daily Report - Sep 22

 

2. Weekly Report & Branch Filtering

  • timeframe: Sep 13-20 (1 week)
  • interval size: 8h (28800s)
  • branches: mozilla-central, try
  • url: <hostname>/reports/pushes?starttime=1284361200&endtime=1284966000&int_size=28800&branch=mozilla-central,try

Notes:

  • busier in the middle of the week, less busy on weekends (as expected)
  • compare only branches of interest/relevant for a more clear view

Weekly Report - Sep 13-20, filter branches: m-c, try

 

All reports have column chart versions:

Weekly Report - Sep 13-20, filter branches: m-c, try (column chart version)

 

3. Monthly Report

  • timeframe: July 1-31 (1 month)
  • interval size: 1 day (86400s)
  • branches: all
  • url: <hostname>/reports/pushes?starttime=1277967600&endtime=1280559600&int_size=86400

Notes:

  • you can distinguish the weeks apart within the month, with little activity on weekends
  • the gap at the beginning of the chart is the Summit week 🙂

Monthly Report - July

 

4. Annual Report

  • timeframe: June-Sep (4 months – we only have data as of June)
  • interval size: 30 days (2592000s)
  • branches: all
  • url: <hostname>/reports/pushes?starttime=1275375600&endtime=1285830000&int_size=2592000

Notes:

  • compare the activity from a moth to another
  • the interval is exactly 30 days, and not the number of days in that month!

Annual Report - June, July, August, September

See Also: Pushes Query, Wait Times Report

Written by Anamaria Stoica

October 16, 2010 at 9:33 am

Posted in Buildapi, Mozilla

Tagged with , ,

Pushes Query

with 2 comments

One other very important piece of information that can be extracted from the Scheduler Database, besides build requests and jobs, are pushes.

The information about one push is spread among 3 tables: sourcestams, sourcestamp_changes and changes.

The SQLAlchemy query fetches all pushes in a specific time frame, and allows filtering and exclusion of specific branches:

s = meta.scheduler_db_meta.tables[‘sourcestamps’]
sch = meta.scheduler_db_meta.tables[‘sourcestamp_changes’]
c = meta.scheduler_db_meta.tables[‘changes’]

q = select([s.c.revision, s.c.branch, c.c.author, c.c.when_timestamp],
and_(sch.c.changeid == c.c.changeid, s.c.id == sch.c.sourcestampid))
q = q.group_by(c.c.when_timestamp, s.c.branch)

# 3. exlude branches – not of interest / fake
# 4. filter branches
# 5. timeframe

Query explained:

  1. JOIN between sourcestamps, sourcestamp_changes and changes tables. The sourcestamps table contains information about the revision, branch, author and the changes table contains information about the change’s timestamp (when_timestamp).
  2. GROUP BY – next we group by the change’s timestamp (multiple builds of the same push will have the same when_timestamp) and branch (one push could affect multiple branches).
  3. Exclude branches that are not of interest like l10n and fake branches like addontester or the ones generated by unittests or talos tests (e.g. mozilla-central-win32-debug-unittest or mozilla-central-macosx64-talos).
  4. Fetch only pushes of requested branches (e.g. mozilla-central, try).
  5. Fetch only pushes having the change’s timestamp (c.when_timestamp) in the requested time frame, specified by starttime and endtime.

See Also: Build Request Query, Wait Times Query

Written by Anamaria Stoica

October 15, 2010 at 4:51 am

Posted in Buildapi, Mozilla

Tagged with , ,