You are not logged in.

1

Thursday, January 19th 2012, 9:25pm

QTtableView slows down when a lot of data is inserted

I have a QTableView thats using a model i made which extends the QAbstractTableModel. The model uses a

Source code

1
QList< QVector<QString> * > 


as the collection. The table view is used to display logging
messages from the application so the collection will eventually get
extremely large... After it inserts a few thousand rows i notice the
table view starts to slow down a lot and eventually the view freezes for
a few seconds before refreshing.. Is it the type of collection im using
thats making it slow down soo much? Is there a better way to store the
data thats being inserted? Does the QTableView support a large amount
of data?

2

Thursday, January 19th 2012, 10:34pm

you've got a whole implementation of model/view, you show us one container and ask why it's slow? er...

QList< QVector<QString> * > isn't a 'heavy' type, so that wont be the problem.

The slowness will be purely down to your code.

This is very responsive

Source code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
#include <QWidget>
#include <QVBoxLayout>
#include <QApplication>

#include <QTableWidget>

int main(int argc, char* argv[])
{
  QApplication app(argc, argv);
 
  QWidget w;
  QVBoxLayout* layout = new QVBoxLayout();

  QTableWidget* table = new QTableWidget();

  table->setRowCount(5000);
  table->setColumnCount(1);

  for(int i = 0; i < 5000; ++i)
    table->setItem(i, 0, new QTableWidgetItem("foo bar baz"));

  layout->addWidget(table);
  w.setLayout(layout);
  w.show();
  app.exec();

  return 0;
}
If you have a problem, CUT and PASTE your code. Do not retype or simplify it. Give a COMPLETE and COMPILABLE example of your problem. Otherwise we are all guessing the problem from a fabrication where relevant details are often missing.

3

Thursday, January 19th 2012, 11:49pm

UPDATED CODE

Sorry...

Heres my model:

TableItemModel.h:

Source code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
#ifndef TABLEITEMMODEL_H
#define TABLEITEMMODEL_H

#include <QAbstractTableModel>
#include <QList>
#include <QString>
#include <QVector>
#include <iostream>
#include "plumber/LogItem.h"

#define DATE_COLUMN 0
#define MODULE_COLUMN 1
#define FILE_AND_LINE_COLUMN 2
#define LEVEL_AND_MESSAGE_COLUMN 3
#define ERROR_COLUMN 4

#define DATE_HEADING "Date"
#define MODULE_HEADING "Module"
#define FILE_AND_LINE_HEADING "File : Line"
#define LEVEL_AND_MESSAGE_HEADING "Level : Message"

#define COLUMN_COUNT 4

class TableItemModel : public QAbstractTableModel {
    Q_OBJECT

public:
     TableItemModel(QObject *parent=0);
     TableItemModel(QVector<LogItem> logItemList, QObject *parent=0);

     int rowCount(const QModelIndex &parent) const;
     int columnCount(const QModelIndex &parent) const;
     QVariant data(const QModelIndex &index, int role) const;
     QVariant headerData(int section, Qt::Orientation orientation, int role) const;
     QVector<LogItem> getList();
     void addLogItem(LogItem logItem);
     void removeAll();
     void removeTopRow();

 private:
     QVector<LogItem> logItemsList;
 };

#endif // TABLEITEMMODEL_H


TableItemModel.cpp

Source code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
#include "TableItemModel.h"

#include <QColor>

TableItemModel::TableItemModel(QObject *parent)
     : QAbstractTableModel(parent)
 {
 }

  TableItemModel::TableItemModel(QVector<LogItem> logItemlist, QObject *parent)
      : QAbstractTableModel(parent)
  {
      logItemsList=logItemlist;
  }

 int TableItemModel::rowCount(const QModelIndex &parent) const
 {
     Q_UNUSED(parent);
     return logItemsList.count();
 }

 int TableItemModel::columnCount(const QModelIndex &parent) const
 {
     Q_UNUSED(parent);
     return COLUMN_COUNT;
 }

 QVariant TableItemModel::data(const QModelIndex &index, int role) const
 {
     if (!index.isValid())
         return QVariant();

     if (index.row() >= logItemsList.size() || index.row() < 0)
         return QVariant();

     if (role == Qt::DisplayRole)
     {

         LogItem item = logItemsList.at(index.row());

         if (index.column() == DATE_COLUMN)
             return QString::fromStdString(item.date);
         else if (index.column() == MODULE_COLUMN)
             return QString::fromStdString(item.module);
         else if (index.column() == FILE_AND_LINE_COLUMN)
             return QString::fromStdString(item.fileAndLine);
         else if (index.column() == LEVEL_AND_MESSAGE_COLUMN)
             return QString::fromStdString(item.levelAndMessage);
     }
     else if (role == Qt::BackgroundColorRole)
     {
         LogItem item = logItemsList.at(index.row());

         if (item.error)
         {
             return QVariant(QColor(Qt::red));
         }
     }

     return QVariant();
 }

 QVariant TableItemModel::headerData(int section, Qt::Orientation orientation, int role) const
 {
     if (role != Qt::DisplayRole)
         return QVariant();

     if (orientation == Qt::Horizontal)
     {
         if (section == DATE_COLUMN)
             return tr(DATE_HEADING);
         else if (section == MODULE_COLUMN)
             return tr(MODULE_HEADING);
         else if (section == FILE_AND_LINE_COLUMN)
             return tr(FILE_AND_LINE_HEADING);
         else if (section == LEVEL_AND_MESSAGE_COLUMN)
             return tr(LEVEL_AND_MESSAGE_HEADING);
     }

     return QVariant();
 }

 QVector<LogItem> TableItemModel::getList()
 {
     return logItemsList;
 }

 void TableItemModel::addLogItem(LogItem logItem)
 {
    int row = logItemsList.count();

    // insert 1 row.
    beginInsertRows(QModelIndex(), row, row);

    logItemsList.append(logItem);

    endInsertRows();
 }

 void TableItemModel::removeAll()
 {
    beginRemoveRows(QModelIndex(), 0, logItemsList.count() - 1);
    logItemsList.clear();
    endRemoveRows();
 }

 void TableItemModel::removeTopRow()
 {
    beginRemoveRows(QModelIndex(), 0, 0);
    logItemsList.remove(0);
    endRemoveRows();
 }


Here is my view:

TableLogger.h

Source code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
#ifndef TABLELOGGER_H
#define TABLELOGGER_H

#include <pthread.h>
#include <QtGui>
#include <log/Logger.h>
#include <QPushButton>
#include <QFrame>
#include <QWidget>

#include "TableItemModel.h"

class TableLogger : public QFrame, public Logger {
    Q_OBJECT

private:
    QTableView *tableView;
    QList< QVector<QString> * > buffer;
    QPushButton* clearButton;

public:
    TableLogger(QWidget *parent, Qt::WindowFlags f=Qt::Window);
    virtual ~TableLogger() throw();
    void showLogWindow();
    void navigateToBottom();
    virtual void enter(const char* file, int line,
                       const Date& date, pthread_t thread,
                       bool error, int level,
                       const std::string& message,
                       const LogController* pipe,
                       const LogEnvironment* object) throw();
private slots:
    void clearButtonPressed();

signals:
    void itemAdded();


}; // end of TableLogger class.

#endif // TABLELOGGER_H


TableLogger.cpp

Source code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
#include "TableLogger.h"

#include <QVector>
#include <QString>
#include <sstream>
#include <QHeaderView>
#include <stdio.h>
#include <log/LogController.h>
#include <log/LogEnvironment.h>
#include <time.h>
#include <LogItem.h>


TableLogger::TableLogger(QWidget* parent, Qt::WindowFlags f) {
    this->setParent(parent);
    this->setWindowFlags(f);

    // resize the log window.
    this->resize(900, 500);
    this->setWindowTitle(QString::fromStdString("Logger"));

    QGridLayout* layout = new QGridLayout();

    // create table view.
    tableView = new QTableView(this);
    tableView->setModel(new TableItemModel(tableView));

    // set layout and add table view.
    this->setLayout(layout);
    layout->addWidget(tableView, 0, 0, 1, 5);

    // resize column headers.
    for (int i = 0; i < tableView->horizontalHeader()->count(); i++)
    {
        tableView->horizontalHeader()->setResizeMode(i, QHeaderView::Stretch);
    }

    // hide the vertical headers.
    tableView->verticalHeader()->setVisible(false);

    clearButton = new QPushButton("Clear Logs", this);
    layout->addWidget(clearButton, 1, 2, 1, 1, Qt::AlignCenter);
    connect(clearButton, SIGNAL(clicked()), this, SLOT(clearButtonPressed()));
}

TableLogger::~TableLogger() throw() {
}

void TableLogger::showLogWindow() {
    this->show();
    tableView->scrollToBottom();
}

void TableLogger::enter(const char *file, int line,
                         const Date &date, pthread_t thread,
                         bool error, int level,
                         const std::string &message,
                         const LogController *pipe,
                         const LogEnvironment *object) throw () {
    Q_UNUSED(object);
    Q_UNUSED(thread);

    // Create new row entry for the table view.
    LogItem item;

    // Get date information.
    item.date = date.format();

    // Get pipe information.
    item.module = pipe->getLogControllerName();

    // Get file Information
    std::string filename(file);
    size_t slash = filename.rfind('/');
    if(slash == filename.npos) slash = -1;

    std::stringstream fileAndLine;
    fileAndLine << filename.substr(slash+1) << " : " << line;
    item.fileAndLine = fileAndLine.str();

    // create the level and message field value;
    std::stringstream levelAndMessage;
    levelAndMessage << level << " : " << message;
    item.levelAndMessage = levelAndMessage.str();

    // Add the error column.
    item.error = error;

    // Get the table model and add the new log item to it.
    TableItemModel *model = static_cast<TableItemModel*>(tableView->model());

    if (model->getList().count() < 5000)
    {
        model->addLogItem(item);
    }
    else
    {
        model->removeTopRow();
        model->addLogItem(item);
    }

    emit itemAdded();
}

void TableLogger::clearButtonPressed()
{
    TableItemModel *model = static_cast<TableItemModel*>(tableView->model());
    model->removeAll();
}

void TableLogger::navigateToBottom()
{
    if (tableView->verticalScrollBar()->sliderPosition() != tableView->verticalScrollBar()->maximum())
    {
        tableView->scrollToBottom();
    }
}


So i have the table logger as a QFrame with a QTableView in it. The enter method in TableLogger.cpp is where its adding new rows. In the item model you'll notice im only displaying 4 columns but the vectors are storing 5 values. the last value isnt displayed. i just needed it so i can change the row color if that 5th column has a certain value.

This post has been edited 2 times, last edit by "muranaka" (Jan 30th 2012, 10:57pm)


4

Friday, January 20th 2012, 8:30am

the string manips shouldnt be too much trouble for modern cpu and compilers, even though you have a fair amount of mix between stl and qt that could probably be simplified.

addLogItem(vector) looks simple enough.

It might be that you get these though too fast? Have you tried e.g. batching the logs - only add 10 or 100 at a time?

Have you built it in release mode and tested the performance?
Have you used a profiler?
If you have a problem, CUT and PASTE your code. Do not retype or simplify it. Give a COMPLETE and COMPILABLE example of your problem. Otherwise we are all guessing the problem from a fabrication where relevant details are often missing.

Junior

Professional

  • "Junior" is male

Posts: 1,623

Location: San Antonio, TX USA

Occupation: Senior Secure Systems Engineer

  • Send private message

5

Wednesday, January 25th 2012, 5:25pm

muranaka,

Noticed you do a file stat. I have ran across a derogation of performance when used in a loop like this on systems. I would test this just to see if maybe it might be the bottle neck your looking for.