Hydra← All projects

QA metrics

Quality

Production Releases
N/A
Production Defect Leakage
N/A
Average Defect Fix Time
N/A

Test Coverage

1 week2 weeks1 month2 months

Automation Coverage

MusesAcademy Product - Total test case automation coverage by platform · trend over 7 days

All tests

Automated share of all test cases
All platforms
59.76%
59.8%57.6%
16/1023/10
245 of 410 automated
mobile
50.79%
52.4%47.1%
16/1023/10
97 of 191 automated
web
67.58%
67.9%65.5%
16/1023/10
148 of 219 automated

Automatable tests

Coverage counting only tests tagged “automatable”
All platforms
62.76%
62.8%61.6%
16/1023/10
214 of 341 automated
mobile
51.23%
53.2%50.8%
16/1023/10
83 of 162 automated
web
73.18%
73.2%70.4%
16/1023/10
131 of 179 automated

Split by Testomat tag — cases without a platform tag aren't counted. “mobile” covers both iOS and Android.

Test Cases Traction

Draft backlog

Drafts: tagged “draft” now, however old · trend over 7 days
Tests in draft
60
6042
16/1023/10
42 older than 7 days · 0 older than 30 days · oldest 20 days

Updated existing test cases

Share of tests created over 7 days ago, not tagged “draft”, that were edited since · trend over 7 days
All tests
22.91%
53.2%22.2%
16/1023/10
104 of 454 updated, last 7 days

Updated = Testomat's last-edit time falls in the window, so a test edited twice counts once. Changes from an import from code count too.

Out of sync

Automated tests edited in Testomat since their last import from code · trend over 7 days
Automated tests
6
63
16/1023/10
of 262 automated, out of sync with the code

Testomat's own “Out of sync”: the code still has the old version until the test is reimported from code or the edit is rejected in Testomat.

Test Execution

1 week2 weeks1 month2 months

Test runs

MusesAcademy Product - Finished runs labelled web or mobilePlatform (iOS / Android) · last 60 days, trend per 24 h

Runs and success rate

Success rate = passed of the tests that ran in finished runs; skipped tests didn't run and are left out
Runs
All platforms
248
180
25/0823/10
last 60 days
Android
19
120
25/0823/10
last 60 days
iOS
55
90
25/0823/10
last 60 days
web
174
150
25/0823/10
last 60 days
Success rate
All platforms
89.4%
100.0%0.0%
25/0823/10
18,796 of 21,013 tests passed, 247 skipped, last 60 days
Android
29.7%
52.8%0.0%
26/0810/09
47 of 158 tests passed, last 60 days
iOS
82.3%
100.0%27.8%
26/0821/10
1,072 of 1,302 tests passed, 169 skipped, last 60 days
web
90.4%
100.0%0.0%
25/0823/10
17,677 of 19,553 tests passed, 78 skipped, last 60 days

Flaky rate

Flaky = failed, then passed on a retry, of the tests that ran in finished runs
All platforms
0.55%
2.4%0.0%
25/0823/10
116 of 21,013 tests passed only on a retry, last 60 days
Android
0.00%
0.0%0.0%
26/0810/09
0 of 158 tests passed only on a retry, last 60 days
iOS
0.31%
3.5%0.0%
26/0821/10
4 of 1,302 tests passed only on a retry, last 60 days
web
0.57%
2.5%0.0%
25/0823/10
112 of 19,553 tests passed only on a retry, last 60 days

Testomat keeps one result per test even when it was retried: one that failed and then passed is marked “Success after retry” and counts as passed in the success rate too. A suite that doesn't retry failed tests never shows any.

Average run duration

Passed and failed runs labelled Regression / Smoke, launch to finish
regression
Android
20.7 min
587.7
31/0810/09
6 runs, last 60 days
iOS
22.2 min
55.51.5
26/0821/10
46 runs, last 60 days
web
11 min
30.51.7
25/0823/10
102 runs, last 60 days
smoke
Android
—
No runs in the last 60 days
iOS
—
No runs in the last 60 days
web
10 min
65.43.1
25/0823/10
58 runs, last 60 days