Ranter
Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Comments
-
php, max. 6 minutes with integration and functional tests, small website with custom cms backend
-
Size in loc is not the only criteria. Fwiw it's prolly the least impacting criteria of all.
How throughout are the tests? Are they just unit, or IT as well? E2e?
Unit - how good is the coverage? How much is reflection used [mockito]?
IT - network, databases, external services, other infra
e2e - network, databases, ext services, flow length, duration, complexity, other infra
perf -- count of test sets, duration of each, infra performance, aggregator/analyzer
endurance -- all of the above
applies to all the tests: how is the state restored? Simple deletes? Instance terminations? Db restores? How often is the qa infra restarted during the tests? Warmup/cooldown periods?
these are some of the criteria impacting tests duration. You could have 10M loc project and only 3 unittests.
And saying just 'we run tests' is very vague. There are many kinds of tests, each with its own criterias impacting the overall duration -
@netikras
I am, of course, aware of that. However, LoC/SLoC count is at least a basic size measurement.
In our case, we have roughly
- 9000 unit tests
- 200 integration tests
- 60 UI tests
(Client - Server application)
I've seen with tests running 400 hours, or even 2 weeks. I'm more interested in the order of magnitude you'll find "in the wild" ;-) -
matste6425yCurrently: 40kloc - 9minutes for all tests (sequential run). 85% unit tests coverage.
In previous job: 1Mloc(horrible Java/Hibernate/JSF monolith): 6hours, the tests were never green, 60% unit tests coverage.
Related Rants
<Question>
I'm curious: For those of you who have tests running in CI pipelines at work:
How long does it take to run the tests, in relation to the system's size?
At my company, it's ~ 30 mins, testing 1M lines of code (750k SLOC) written in Java (85%) and JS (15%).
question
performance
build
pipeline
ci
tests