Sumac to Teak upgrade fail

After upgrading Sumac to Teak, and running

```tutor local upgrade –from=sumac```

the next time I run a `tutor local do init`, I get 3 errors:

```

19 of 30 ERROR creating sql materialized_view model reporting.fact_video_segments [ERROR in 0.09s]

27 of 30 ERROR creating sql materialized_view model reporting.dim_problem_coursewide_avg [ERROR in 0.03s]

28 of 30 ERROR creating sql materialized_view model reporting.dim_subsection_problem_results [ERROR in 0.11s]

any idea what this may be ?

```

22:43:12  Finished running 2 dictionary models, 22 materialized view models, 6 view models in 0 hours 0 minutes and 2.27 seconds (2.27s).
22:43:12
22:43:12  Completed with 3 errors, 0 partial successes, and 0 warnings:
22:43:12
22:43:12    Database Error in model fact_video_segments (models/video/fact_video_segments.sql)
  Code: 47.
  DB::Exception: Unknown expression identifier `emission_time_long` in scope data AS (SELECT event_id, org, course_key, actor_id, emission_time_long, video_position, object_id, video_duration, if(verb_id = 'https://w3id.org/xapi/video/verbs/played', 'start', 'end') AS verb FROM xapi.video_playback_events WHERE verb_id != 'http://adlnet.gov/expapi/verbs/initialized'). Maybe you meant: ['emission_time']. Stack trace:

  0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000133b761f
  1. DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000c8623ce
  2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000c861e80
  3. DB::Exception::Exception<char const*, String&, String, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String&>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&&, String&, String&&, String&&, String&&) @ 0x0000000017430702
  4. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173fc0ff
  5. DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000173f9374
  6. DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&) @ 0x00000000174314a7
  7. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efdee
  8. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173f9eda
  9. DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017432458
  10. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efda4
  11. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173f9eda
  12. DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017432458
  13. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efda4
  14. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173f9eda
  15. DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017432458
  16. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efda4
  17. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173f9eda
  18. DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017432458
  19. DB::QueryAnalyzer::resolveJoin(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017433c7a
  20. DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017431ef8
  21. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efda4
  22. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173f9eda
  23. DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017432458
  24. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efda4
  25. DB::QueryAnalyzer::resolve(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::IQueryTreeNode> const&, std::shared_ptr<DB::Context const>) @ 0x00000000173ed738
  26. DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x00000000173ecd9c
  27. DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x0000000017457636
  28. DB::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) (.llvm.4471323483497696151) @ 0x0000000017fb598d
  29. DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000017fb38dc
  30. DB::InterpreterSelectQueryAnalyzer::getSampleBlock(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&) @ 0x0000000017fb5de1
  31. DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&, DB::LoadingStrictnessLevel) const @ 0x0000000017ed75c7
22:43:12
22:43:12    Database Error in model dim_problem_coursewide_avg (models/problems/dim_problem_coursewide_avg.sql)
  Code: 47.
  DB::Exception: Unknown expression identifier `problem_link` in scope last_response AS (SELECT org, course_key, problem_link, problem_id, actor_id, course_order, graded, multiIf(success, attempts, 0) AS success_attempt, multiIf(NOT success, attempts, 0) AS incorrect_attempt FROM reporting.dim_learner_last_response). Maybe you meant: ['problem_link']. Stack trace:

  0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000133b761f
  1. DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000c8623ce
  2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000c861e80
  3. DB::Exception::Exception<char const*, String&, String, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String&>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&&, String&, String&&, String&&, String&&) @ 0x0000000017430702
  4. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173fc0ff
  5. DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000173f9374
  6. DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&) @ 0x00000000174314a7
  7. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efdee
  8. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173f9eda
  9. DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017432458
  10. DB::QueryAnalyzer::resolveJoin(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017433c7a
  11. DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017431ef8
  12. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efda4
  13. DB::QueryAnalyzer::resolve(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::IQueryTreeNode> const&, std::shared_ptr<DB::Context const>) @ 0x00000000173ed738
  14. DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x00000000173ecd9c
  15. DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x0000000017457636
  16. DB::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) (.llvm.4471323483497696151) @ 0x0000000017fb598d
  17. DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000017fb38dc
  18. DB::InterpreterSelectQueryAnalyzer::getSampleBlock(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&) @ 0x0000000017fb5de1
  19. DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&, DB::LoadingStrictnessLevel) const @ 0x0000000017ed75c7
  20. DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000017edfe3f
  21. DB::InterpreterCreateQuery::execute() @ 0x0000000017ef63f8
  22. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, std::unique_ptr<DB::ReadBuffer, std::default_delete<DB::ReadBuffer>>&, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::ImplicitTransactionControlExecutor>) @ 0x0000000018385b92
  23. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000001837de0b
  24. DB::TCPHandler::runImpl() @ 0x0000000019a5a0ca
  25. DB::TCPHandler::run() @ 0x0000000019a7c119
  26. Poco::Net::TCPServerConnection::start() @ 0x000000001efa03c7
  27. Poco::Net::TCPServerDispatcher::run() @ 0x000000001efa0859
  28. Poco::PooledThread::run() @ 0x000000001ef66e47
  29. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001ef65241
  30. ? @ 0x0000000000094ac3
  31. ? @ 0x0000000000125a74
22:43:12
22:43:12    Database Error in model dim_subsection_problem_results (models/problems/dim_subsection_problem_results.sql)
  Code: 47.
  DB::Exception: Identifier 'last_response.display_name_with_location' cannot be resolved from table with name last_response. In scope SELECT last_response.org AS org, last_response.course_key AS course_key, last_response.actor_id AS actor_id, splitByChar('@', subsection_blocks.block_id)[3] AS block_id_short, last_response.problem_id AS problem_id, last_response.success AS success, last_response.scaled_score AS scaled_score, splitByString(' - ', subsection_blocks.display_name_with_location) AS _subsection_with_name, arrayStringConcat(arrayMap(x -> leftPad(x, 2, char(917768)), splitByString(':', _subsection_with_name[1])), ':') AS subsection_number, concat(subsection_number, ' - ', _subsection_with_name[2]) AS subsection_with_name, substring(regexpExtract(last_response.object_id, '(@problem\\+block@[\\w\\d][^_/]*)(_\\d)?', 2), 2) AS _problem_id_number, ifNull(nullIf(_problem_id_number, ''), '1') AS _problem_id_or_1, splitByString(' - ', last_response.display_name_with_location) AS _problem_with_name, arrayStringConcat(arrayMap(x -> leftPad(x, 2, char(917768)), splitByString(':', _problem_with_name[1])), ':') AS _problem_number, concat(_problem_number, '_', _problem_id_or_1) AS problem_number, concat(problem_number, ' - ', _problem_with_name[2]) AS problem_name_location FROM reporting.dim_learner_last_response AS last_response INNER JOIN reporting.dim_course_blocks AS problem_blocks ON (problem_blocks.block_id = last_response.problem_id) AND (last_response.course_key = problem_blocks.course_key) LEFT JOIN reporting.dim_course_blocks AS subsection_blocks ON (problem_blocks.subsection_number = subsection_blocks.hierarchy_location) AND (last_response.org = subsection_blocks.org) AND (last_response.course_key = subsection_blocks.course_key) AND ((subsection_blocks.block_id LIKE '%@sequential+block@%') OR (subsection_blocks.block_id LIKE '%@chapter+block@%')) WHERE last_response.graded. Stack trace:

  0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000133b761f
  1. DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000c8623ce
  2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000c861e80
  3. DB::Exception::Exception<String const&, String&, String, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String const&, String&, String&&, String&&) @ 0x0000000017671330
  4. DB::IdentifierResolver::tryResolveIdentifierFromStorage(DB::Identifier const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::AnalysisTableExpressionData const&, DB::IdentifierResolveScope&, unsigned long, bool) @ 0x0000000017670b6e
  5. DB::IdentifierResolver::tryResolveIdentifierFromTableExpression(DB::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000176715e2
  6. DB::IdentifierResolver::tryResolveIdentifierFromJoinTreeNode(DB::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000176720ba
  7. DB::IdentifierResolver::tryResolveIdentifierFromJoin(DB::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000176725c2
  8. DB::IdentifierResolver::tryResolveIdentifierFromJoinTreeNode(DB::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000176720e6
  9. DB::IdentifierResolver::tryResolveIdentifierFromJoin(DB::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000176725c2
  10. DB::IdentifierResolver::tryResolveIdentifierFromJoinTreeNode(DB::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000176720e6
  11. DB::IdentifierResolver::tryResolveIdentifierFromJoinTree(DB::IdentifierLookup const&, DB::IdentifierResolveScope&) @ 0x0000000017676a8d
  12. DB::QueryAnalyzer::tryResolveIdentifier(DB::IdentifierLookup const&, DB::IdentifierResolveScope&, DB::IdentifierResolveContext) @ 0x000000001740cad3
  13. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173fa501
  14. DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000173f9374
  15. DB::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&) @ 0x000000001741501b
  16. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173f9fa0
  17. DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool) @ 0x00000000173f9374
  18. DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&) @ 0x00000000174314a7
  19. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efdee
  20. DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x00000000173f9eda
  21. DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x0000000017432458
  22. DB::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::IdentifierResolveScope&) @ 0x00000000173efda4
  23. DB::QueryAnalyzer::resolve(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::IQueryTreeNode> const&, std::shared_ptr<DB::Context const>) @ 0x00000000173ed738
  24. DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x00000000173ecd9c
  25. DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x0000000017457636
  26. DB::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) (.llvm.4471323483497696151) @ 0x0000000017fb598d
  27. DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000017fb38dc
  28. DB::InterpreterSelectQueryAnalyzer::getSampleBlock(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&) @ 0x0000000017fb5de1
  29. DB::AlterCommand::apply(DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context const>) const @ 0x000000001896ec07
  30. DB::AlterCommands::apply(DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context const>) const @ 0x0000000018974af1
  31. DB::StorageMaterializedView::alter(DB::AlterCommands const&, std::shared_ptr<DB::Context const>, std::unique_lock<std::timed_mutex>&) @ 0x0000000018af2b3a
  compiled code at target/run/aspects/models/problems/dim_subsection_problem_results.sql
22:43:12
22:43:12  Done. PASS=25 WARN=0 ERROR=3 SKIP=2 TOTAL=30

This seems related to aspects/clickhouse, which I also upgraded from 2.3.1 to 2.5.0

Hi @mboisson can you please try out running
tutor local do dbt --only-changed=false -c "run --full-refresh --select dim_blah dim_blah" it might resolve the issues you are facing

@mboisson did you ever resolve this issue?

Yes, thank you. The issue was with Aspect that needed to run some scripts.

@mboisson for people finding this thread later, could you detail what you did and mark it as the solution?

Sure. The solution was to follow the uprade instructions for Aspect 2.5.0. In particular, running

```tutor local do dbt --only_changed False -c ‘run --full-refresh --select dim_learner_last_response dim_problem_results problem_events dim_subsection_problem_results’```

1 Like