Metric refresh performance improvement (nosync)

The SQLITE database syncs (writes) to the metricDB file after
every atomic transaction. As the size of the database grows this
becomes very expensive (suspect it is writing the entire DB).

To avoid this behaviour the Metric refresh code opens and commits
a LUW for the entire refresh. This has shown to improve the performance
of the metric refresh quite dramatically (50% reduction in overall
time to complete a metric refresh).

Big props to Fredrik Lingvall for detailed analysis and profiling
of the code to identify the inefficiencies.
This commit is contained in:
Mark Liversedge
2011-05-12 20:13:55 +01:00
parent 3ae81329d7
commit d29d97231c

View File

@@ -81,6 +81,9 @@ void MetricAggregator::refreshMetrics()
dbStatus.insert(filename, add);
}
// begin LUW -- byproduct of turning off sync (nosync)
dbaccess->connection().transaction();
// Delete statistics for non-existant ride files
QHash<QString, status>::iterator d;
for (d = dbStatus.begin(); d != dbStatus.end(); ++d) {
@@ -170,6 +173,9 @@ void MetricAggregator::refreshMetrics()
out << "METRIC REFRESH ENDS: " << QDateTime::currentDateTime().toString() + "\r\n";
log.close();
// end LUW -- now syncs DB
dbaccess->connection().commit();
main->isclean = true;
dataChanged(); // notify models/views