# Course import issue from ironwood to juniper

**URL:** <https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811>\
**Category:** Site Operations Help\
**Tags:** how-to, juniper\
**Created:** [December 11, 2020, 10:08am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811 "2020-12-11T10:08:14Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zubair](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zubair/32/1720_2.png) [@Zubair](https://discuss.openedx.org/u/Zubair)\
**Post date:** [December 11, 2020, 10:08am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/1 "2020-12-11T10:08:14Z")

</div>

I’m trying to import a course that has multiple videos and is hosted on an ironwood server to a newly deployed juniper server. While importing it throws the following errors in the console.  
Following is the console log output

```auto
TypeError: Unicode-objects must be encoded before hashing
[2020-12-11 09:29:02,847: INFO/Worker-1] VAL: Video created with id [54454916-47e9-4769-8b41-06062d0b7e8c] and status [external]
[2020-12-11 09:29:02,860: ERROR/Worker-1] [VAL] Transcript save failed to storage for video_id "54454916-47e9-4769-8b41-06062d0b7e8c" language code "en"
Traceback (most recent call last):
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/edxval/models.py", line 489, in create
    video_transcript.transcript.save(file_name, transcript_content)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/django/db/models/fields/files.py", line 87, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 495, in _save
    self._save_content(obj, content, parameters=parameters)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 510, in _save_content
    obj.upload_fileobj(content, ExtraArgs=put_parameters)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/boto3/s3/inject.py", line 513, in object_upload_fileobj
    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/boto3/s3/inject.py", line 431, in upload_fileobj
    return future.result()
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/futures.py", line 73, in result
    return self._coordinator.result()
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/futures.py", line 233, in result
    raise self._exception
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/upload.py", line 692, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/client.py", line 317, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/client.py", line 596, in _make_api_call
    request_signer=self._request_signer, context=request_context)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/hooks.py", line 242, in emit_until_response
    responses = self._emit(event_name, kwargs, stop_on_response=True)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/hooks.py", line 210, in _emit
    response = handler(**kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/handlers.py", line 209, in conditionally_calculate_md5
    calculate_md5(params, **kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/handlers.py", line 187, in calculate_md5
    binary_md5 = _calculate_md5_from_file(body)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/handlers.py", line 201, in _calculate_md5_from_file
    md5.update(chunk)

```

---

<div class="post-metadata">

**Author:** ![Zubair](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zubair/32/1720_2.png) [@Zubair](https://discuss.openedx.org/u/Zubair)\
**Post date:** [December 11, 2020, 4:33pm UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/2 "2020-12-11T16:33:41Z")

</div>

@regis can you please look into this and tell me what is happening? I have tried updating botcore version but same error occurs. I saw you comment [here](https://github.com/overhangio/tutor/issues/7#issuecomment-370256117), I’m getting similar errors in the logs

```auto
[2020-12-11 09:29:05,993: WARNING/Worker-1] XBlock <class 'xblock.internal.DiscussionXBlockWithMixins'> does not contain field url_name
[2020-12-11 09:29:05,994: INFO/Worker-1] Exception 'str' object has no attribute ' __traceback__' when trying to load definition xml for block <DiscussionXBlockWithMixins @ECEE discussion_target='1. 1A page1', source_file=None, tags=[], xqa_key=None, discussion_category='Unit 1: The integral', user_partitions=[], matlab_api_key=None, static_asset_path='', format=None, use_latex_compiler=False, due=None, edxnotes_visibility=True, max_attempts=None, self_paced=False, days_early_for_beta=None, discussion_id='discussion_psA_1-tab1', sort_key=None, start=datetime.datetime(2030, 1, 1, 0, 0, tzinfo=<UTC>), video_auto_advance=False, chrome=None, group_access={}, showanswer='finished', show_reset_button=False, in_entrance_exam=False, name=None, video_speed_optimizations=True, graceperiod=None, default_tab=None, hide_from_toc=False, show_correctness='always', graded=False, video_bumper={}, display_name='1. 1A page1', course_edit_method='Studio', xml_attributes={}, parent=None, edxnotes=False, visible_to_staff_only=False, rerandomize='never', giturl=None> - assuming XBlock export format

```

---

<div class="post-metadata">

**Author:** ![regis](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/regis/32/2640_2.png) [@regis](https://discuss.openedx.org/u/regis)\
**Post date:** [December 11, 2020, 5:17pm UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/3 "2020-12-11T17:17:33Z")

</div>

Indeed I have investigated this issue before, not [there](https://github.com/overhangio/tutor/issues/7#issuecomment-370256117), but [here](https://discuss.overhang.io/t/video-transcript-upload-generates-500-error/836/6).

The solution is provided in [this answer](https://discuss.overhang.io/t/video-transcript-upload-generates-500-error/836/6). tl;dr: you must apply [this patch](https://github.com/nguyennk92/edx-platform/commit/e70cf63180b3acd5e19032235595c880cb35bc20.patch) to edx-platform.

---

<div class="post-metadata">

**Author:** ![Zubair](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zubair/32/1720_2.png) [@Zubair](https://discuss.openedx.org/u/Zubair)\
**Post date:** [December 14, 2020, 5:23am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/4 "2020-12-14T05:23:11Z")

</div>

Bundle of thanks for posting the solution, got stuck on it & couldn’t find any solutions anywhere so far. Thanks again @regis .

---

<div class="post-metadata">

**Author:** ![Zubair](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zubair/32/1720_2.png) [@Zubair](https://discuss.openedx.org/u/Zubair)\
**Post date:** [December 14, 2020, 7:51am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/5 "2020-12-14T07:51:48Z")

</div>

@regis strange but I’m facing same issue after applying the suggested solution, I don’t get this part of the answer though  
`Apply it to Dockerfile and rebuild image:`  
` RUN curl https://github.com/nguyennk92/edx-platform/commit/e70cf63180b3acd5e19032235595c880cb35bc20.patch 9 | git apply -`

---

<div class="post-metadata">

**Author:** ![regis](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/regis/32/2640_2.png) [@regis](https://discuss.openedx.org/u/regis)\
**Post date:** [December 14, 2020, 9:45am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/6 "2020-12-14T09:45:02Z")

</div>

The instructions in the above-mentioned post apply to [Tutor](https://docs.tutor.overhang.io/); if you run the native installation, you should patch edx-platform and [restart the lms service](https://openedx.atlassian.net/wiki/spaces/OpenOPS/pages/60227913/Managing+Open+edX+Tips+and+Tricks#ManagingOpenedXTipsandTricks-RestartingServices).

---

<div class="post-metadata">

**Author:** ![Zubair](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zubair/32/1720_2.png) [@Zubair](https://discuss.openedx.org/u/Zubair)\
**Post date:** [December 14, 2020, 9:46am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/7 "2020-12-14T09:46:49Z")

</div>

Already did that but still getting the error…

---

<div class="post-metadata">

**Author:** ![regis](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/regis/32/2640_2.png) [@regis](https://discuss.openedx.org/u/regis)\
**Post date:** [December 14, 2020, 10:40am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/8 "2020-12-14T10:40:41Z")

</div>

Did you also restart the cms service? (you should)

---

<div class="post-metadata">

**Author:** ![Zubair](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zubair/32/1720_2.png) [@Zubair](https://discuss.openedx.org/u/Zubair)\
**Post date:** [December 14, 2020, 10:44am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/9 "2020-12-14T10:44:26Z")

</div>

@regis Yes I did restart all the services. I even updated `edxval` version to 1.4.2 but still getting this error.  
It throws the following error and the course shows empty verticles for units that contained videos.

```
TypeError: Unicode-objects must be encoded before hashing
[2020-12-11 09:29:02,735: ERROR/Worker-1] Unable to load child when parsing Vertical. Continuing...
Traceback (most recent call last):
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/vertical_block.py", line 169, in definition_from_xml
    child_block = system.process_xml(etree.tostring(child, encoding='unicode'))
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/xml.py", line 173, in process_xml
    id_manager,
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1714, in xblock_from_node
    block = block_class.parse_xml(node, self, keys, id_generator)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 569, in parse_xml
    return super(XmlParserMixin, cls).parse_xml(node, runtime, keys, id_generator) # pylint: disable=bad-super-call
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1156, in parse_xml
    block = cls.from_xml(xml, runtime, id_generator)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/video_module/video_module.py", line 673, in from_xml
    getattr(id_generator, 'target_course_id', None)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/video_module/video_module.py", line 1023, in import_video_info_into_val
    course_id=course_id
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/edxval/api.py", line 1113, in import_from_xml
    create_transcript_objects(xml, edx_video_id, resource_fs, static_dir, external_transcripts)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/edxval/api.py", line 1232, in create_transcript_objects
    static_dir=static_dir
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/edxval/api.py", line 1176, in import_transcript_from_fs
    provider=provider
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/edxval/api.py", line 350, in create_video_transcript
    transcript_serializer.save(content=content)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/rest_framework/serializers.py", line 214, in save
    self.instance = self.create(validated_data)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/edxval/serializers.py", line 94, in create
    return VideoTranscript.create(**validated_data)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/edxval/models.py", line 489, in create
    video_transcript.transcript.save(file_name, transcript_content)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/django/db/models/fields/files.py", line 87, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 495, in _save
    self._save_content(obj, content, parameters=parameters)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 510, in _save_content
    obj.upload_fileobj(content, ExtraArgs=put_parameters)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/boto3/s3/inject.py", line 513, in object_upload_fileobj
    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/boto3/s3/inject.py", line 431, in upload_fileobj
    return future.result()
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/futures.py", line 73, in result
    return self._coordinator.result()
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/futures.py", line 233, in result
    raise self._exception
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/s3transfer/upload.py", line 692, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/client.py", line 317, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/client.py", line 596, in _make_api_call
    request_signer=self._request_signer, context=request_context)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/hooks.py", line 242, in emit_until_response
    responses = self._emit(event_name, kwargs, stop_on_response=True)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/hooks.py", line 210, in _emit
    response = handler(**kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/handlers.py", line 209, in conditionally_calculate_md5
    calculate_md5(params, **kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/handlers.py", line 187, in calculate_md5
    binary_md5 = _calculate_md5_from_file(body)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.5/site-packages/botocore/handlers.py", line 201, in _calculate_md5_from_file
    md5.update(chunk)
```

---

<div class="post-metadata">

**Author:** ![regis](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/regis/32/2640_2.png) [@regis](https://discuss.openedx.org/u/regis)\
**Post date:** [December 14, 2020, 11:54am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/10 "2020-12-14T11:54:37Z")

</div>

Indeed, it appears that the issue still occurs in Koa. There might be a deeper issue with edxval. I re-opened my PR here: [https://github.com/edx/edx-platform/pull/24800#issuecomment-744389775](https://github.com/edx/edx-platform/pull/24800#issuecomment-744389775)

---

<div class="post-metadata">

**Author:** ![Zubair](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zubair/32/1720_2.png) [@Zubair](https://discuss.openedx.org/u/Zubair)\
**Post date:** [December 14, 2020, 11:57am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/11 "2020-12-14T11:57:10Z")

</div>

I was just reading the comments on that `PR`, let’s see. 🙂

---

<div class="post-metadata">

**Author:** ![Zubair](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zubair/32/1720_2.png) [@Zubair](https://discuss.openedx.org/u/Zubair)\
**Post date:** [December 15, 2020, 6:43am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/12 "2020-12-15T06:43:44Z")

</div>

@regis I checked `edxval` release versions and [version 1.4.3](https://github.com/edx/edx-val/releases/tag/1.4.3) is the one that fixed S3 bucket upload issue and it works for me

---

<div class="post-metadata">

**Author:** ![regis](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/regis/32/2640_2.png) [@regis](https://discuss.openedx.org/u/regis)\
**Post date:** [December 15, 2020, 7:04am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/13 "2020-12-15T07:04:25Z")

</div>

@Zubair can you please comment on the PR? This is an issue that should be fixed in Koa.

---

<div class="post-metadata">

**Author:** ![Zubair](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zubair/32/1720_2.png) [@Zubair](https://discuss.openedx.org/u/Zubair)\
**Post date:** [December 15, 2020, 9:44am UTC](https://discuss.openedx.org/t/course-import-issue-from-ironwood-to-juniper/3811/14 "2020-12-15T09:44:43Z")

</div>

Added the comment and tagged you there.