Uploaded image for project: 'UGENE'
  1. UGENE
  2. UGENE-8110

Race condition when closing UGENE and canceling a subtask while opening UGENE

    XMLWordPrintable

    Details

    • Tests Type:
      Untestable
    • Sprint:
      DEV-51-4
    • User Requests Number:
      1
    • Affect Type:
      Userdefined

      Description

      In the crash report 2824, a user closes UGENE while it is opening. The following execution sequence is possible:

      1. [10:33:30.470] [Tasks] Задача {Запуск сервиса: Проект} Инициализирована
        In English: Promoting task {Enable 'Project' service} to 'Prepared'. This task is part of the task of opening a user file in UGENE. This task initializes the Project.
      2. It adds the "Enable Project" task as a subtask (EnableServiceTask::prepare:237). This task is scheduled to run by the UGENE task scheduler.
      3. Then the "Отмена активных задач" ("Cancel active tasks") task comes into play. This task generates the following lines in the report:
        [10:33:30.470] [Core Services] Canceling: Opening document: C:/Users/.../AppData/Local/Temp/...zip.79f/model1.pdb
        [10:33:30.470] [Tasks] Задача {Отмена активных задач} Инициализирована
      4. Canceling the top-level task "Opening document: C:/Users/.../AppData/Local/Temp/...zip.79f/model1.pdb" cancels all its subtasks, including the tasks "Enable 'Project' service" and "Enable Project". The newly added subtask "Enable project" will not run. In the current implementation, this means that none of its methods will be called.
      5. In its ProjectServiceEnableTask::report:139 method, this task sets the project to the global context. But due to the fact that it has been cancelled, this will not be done.
      6. Then the program execution returns to the parent task "Запуск сервиса: Проект" ("Enable 'Project' service"). Since the subtask is considered completed (no errors), the service is enabled and connections are made (EnableServiceTask::report:249).
      7. As a result, the "Project" service is enabled, but no project is defined in the global context. Invariants are violated, SAFE_POINTs occur, connection errors occur:
        [10:33:30.494] [User Interface] QObject::connect: Cannot connect (nullptr)::si_projectURLChanged(const QString&) to U2::ProjectLoaderImpl::sl_projectURLChanged(const QString&)
        [10:33:30.494] [Core Services] Trying to recover from error: QObject.connect: Cannot connect (nullptr)::si_projectURLChanged(const QString&) to U2::ProjectLoaderImpl::sl_projectURLChanged(const QString&) at C:\BuildAgent\work\release\ugene\src\corelibs\U2Core\src\globals\Log.cpp:92
        [10:33:30.494] [User Interface] QObject::connect: Cannot connect (nullptr)::si_documentAdded(Document*) to U2::ProjectLoaderImpl::sl_documentAdded(Document*)
        [10:33:30.494] [Core Services] Trying to recover from error: QObject.connect: Cannot connect (nullptr)::si_documentAdded(Document*) to U2::ProjectLoaderImpl::sl_documentAdded(Document*) at C:\BuildAgent\work\release\ugene\src\corelibs\U2Core\src\globals\Log.cpp:92
        [10:33:30.494] [Tasks] Задача {Запуск сервиса: Проект} Завершена
      8. The services "Навигатор проектов" ("ProjectView"), "Поддержка внешних инструментов" ("External tools support") and "Экспорт ДНК" ("DNA export service") depend on the "Project" service. Enabling the "Project" triggers new top-level tasks:
        [10:33:30.693] [Tasks] Добавлена задача "Запуск сервиса: Навигатор проектов"
        [10:33:30.797] [Tasks] Добавлена задача "Запуск сервиса: Поддержка внешних инструментов"
        [10:33:30.968] [Tasks] Добавлена задача "Запуск сервиса: Экспорт ДНК"
      9. All of them, as well as the original "Opening document: C:/Users/.../AppData/Local/Temp/...zip.79f/model1.pdb" task, assuming a valid project in the global context, generate new SAFE_POINTs and Qt errors in the log:
        [10:33:30.495] [Core Services] Trying to recover from error: No active project found! at C:\BuildAgent\work\release\ugene\src\ugeneui\src\project_support\ProjectLoaderImpl.cpp:1084
        [10:33:30.495] [Tasks] Задача {Loading documents} Завершена
        [10:33:30.866] [Core Services] Trying to recover from error: No project view found at C:\BuildAgent\work\release\ugene\src\plugins\external_tool_support\src\ETSProjectViewItemsController.cpp:54
        [10:33:30.866] [Tasks] Задача {Запуск сервиса: Поддержка внешних инструментов} Завершена
        [10:33:31.138] [User Interface] QObject::connect: Cannot connect (nullptr)::si_onDocTreePopupMenuRequested(QMenu&) to U2::ExportProjectViewItemsContoller::sl_addToProjectViewMenu(QMenu&)
        [10:33:31.138] [Core Services] Trying to recover from error: QObject.connect: Cannot connect (nullptr)::si_onDocTreePopupMenuRequested(QMenu&) to U2::ExportProjectViewItemsContoller::sl_addToProjectViewMenu(QMenu&) at C:\BuildAgent\work\release\ugene\src\corelibs\U2Core\src\globals\Log.cpp:92
        [10:33:31.139] [Tasks] Задача {Запуск сервиса: Экспорт ДНК} Завершена
      10. Also, the "Отмена активных задач" ("Cancel active tasks") task fails: the top-level tasks are still present (the added "Запуск сервиса: ..." ("Enable '...' service") tasks). UGENE is not closed.
        [10:33:30.711] [Tasks] Задача {Завершение программы} Завершена; ошибка: Имеются неоконченные задачи
        [10:33:30.711] [Tasks] Отменяется задача {Завершение программы}
      11. The user notices this after some short time and presses the cross again.
      12. After many places protected by SAFE_POINTs, one was found where UGENE crashed. The "Остановка сервиса: Навигатор проектов" ("Disable 'ProjectView' service") task generated by clicking the cross didn't expect project=null (DisableProjectViewTask::prepare:1187).

      Possible solutions:

      1. Change the approach to opening/closing UGENE. The fact that closing UGENE is separated into a separate cancelable task is a controversial decision. Requires huge refactoring and will introduce potential new bugs.
      2. Revert the 1414 commit. Canceled tasks are considered completed; previously, the report method of their subtasks was called even after cancellation. Not now.
      3. Add checks for cancellations/errors of tasks and subtasks, add setting errors to tasks and subtasks and/or correct task flags (ErrorOnError, ErrorOnCancel). First of all, look at the EnableServiceTask::report:248-249: do not initialize the service if the task is canceled. After the changes, check that it will work correctly for all services. It may be necessary to add CHECK_NN Project wherever AppContext::getProject() is called (or even for all services?). This can also lead to memory leaks: if a Project is allocated on the heap, who will delete it?

      There is no scenario because the following subsequence of log messages is required:
      [Tasks] Задача {Запуск сервиса: Проект} Инициализирована
      [Core Services] Canceling: Opening document: ...
      [Tasks] Задача {Отмена активных задач} Инициализирована
      In fact, the "Запуск сервиса: Проект" ("Enable 'Project' service") subtask always finishes first before the "Отмена активных задач" ("Cancel active tasks") starts, so it cannot be reproduced.

      However, there is the crash report 2839 from me, obtained by "manually forcing" the cancellation of the "Opening document: ..." task at the required timing (so that as in the crash report). The resulting crash report is close to the original one in terms of errors in the log, scenario, reason, address and location of the crash. Added the following lines to the end of the TaskSchedulerImpl::tryPrepare method:

      if (task->getTaskName() == "Enable 'Project' service") {
          for (auto&& t : qAsConst(AppContext::getTaskScheduler()->getTopLevelTasks())) {
              coreLog.trace(tr("Canceling: %1").arg(t->getTaskName()));
              t->cancel();
          }
      }
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              biserova Evelina
              Reporter:
              biserova Evelina
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: