Scorm XBlock handle to serve SCORM content from a private S3 bucket

README.rst

@@ -17,11 +17,11 @@ This XBlock is not compatible with its `ancestor <https://github.com/raccoongang
 Installation
 ------------
 
-This XBlock was designed to work out of the box with `Tutor <https://docs.tutor.overhang.io>`__ (Ironwood release). It comes bundled by default in the official Tutor releases, such that there is no need to install it manually.
+This XBlock was designed to work out of the box with `Tutor <https://docs.tutor.overhang.io>`__ (Ironwood release). But in this fork it has been made compatible with juniper release. It comes bundled by default in the official Tutor releases, such that there is no need to install it manually.
 
-For non-Tutor platforms, you should install the `Python package from Pypi <https://pypi.org/project/openedx-scorm-xblock/>`__::
+For non-Tutor platforms, you can install it using following command::
 
-    pip install openedx-scorm-xblock
+    pip install git+https://github.com/edly-io/openedx-scorm-xblock.git@master#egg=openedx-scorm-xblock
 
 Usage
 -----
@@ -37,16 +37,53 @@ By default, SCORM modules will be accessible at "/scorm/" urls and static assets
 
     XBLOCK_SETTINGS["ScormXBlock"] = {
         "LOCATION": "alternatevalue",
+        "SCORM_FILE_STORAGE_TYPE": "openedx.features.clearesult_features.backend_storage.ScormXblockS3Storage"
     }
 
+Here ``SCORM_FILE_STORAGE_TYPE`` shows that we are using our own custom storage backend. You can get the idea from
+`Philanthropy Storage Backend <https://github.com/philanthropy-u/edx-platform/blob/master/openedx/features/philu_utils/backend_storage.py>`_
+
+If you are using this xblock locally, there is a configuration variable whose value you will have to set empty string::
+
+    XBLOCK_SETTINGS["ScormXBlock"] = {
+        "SCORM_MEDIA_BASE_URL": ""
+    }
+
+You can face x-frame-options restrictions. For that, you can use any workaround. Like for local you can install any extension on chrome which will disable x-frame-restrictions.
+For example: `Ignore X-Frame headers <https://chrome.google.com/webstore/detail/ignore-x-frame-headers/gleekbfjekiniecknbkamfmkohkpodhe>`_ Google Chrome extension.
+
+
 Development
 -----------
 
 Run unit tests with::
 
     $ NO_PREREQ_INSTALL=1 paver test_system -s lms -t openedxscorm
+Nginx settings (for non-local environments) if you are using s3-buckets
+------
+In ``/etc/nginx/sites-enabled/lms`` and ``/etc/nginx/sites-enabled/cms`` put these rules::
+
+    location /scorm/ {
+        rewrite ^/scorm/(.*) /$1 break;
+        try_files $uri @s3;
+      }
+    location @s3 {
+        proxy_pass <YOUR S3 BUCKET BASE PATH>;
+      }
+
+**For exmaple** Your s3 bucket base path can be
+https://c90081bas2001edx.s3.amazonaws.com
+We are doing this because in openedx, we will retrieve content from s3-bucket and display it in an iframe. But because of x-frame-options restrictions we will be blocked. So, to overcome that hurdle we have just replaced the base s3-bucket url with our platform base url in our xblock. Let's understand it from an exmaple. Let's say one of your url for scorm asset available on s3-bucket is
+https://c90081bas2001edx.s3.amazonaws.com/scorm/503a49b7ed1d4a2caa22af84df87fa8c/f792c4c021da74aac780e072bf16aa0ed4987767/shared/launchpage.html
+But openedx will be unable to open it in an iframe. So what we have done is that we have replaced the base url of this url with the base url of openedx so the source url for the iframe will be
+https://dev.learn.clearesult.com/scorm/503a49b7ed1d4a2caa22af84df87fa8c/f792c4c021da74aac780e072bf16aa0ed4987767/shared/launchpage.html
+In this way we overcome the x-frame-options restrictions. Now the other problem is that our scorm asset is not available on this url. It is available on s3-bucket. This is where the nginx rules come handy. When any hit will be made for
+https://dev.learn.clearesult.com/scorm/503a49b7ed1d4a2caa22af84df87fa8c/f792c4c021da74aac780e072bf16aa0ed4987767/shared/launchpage.html
+It will go to nginx where we have already written a rule that for all those requests whose urls start with `/scorm/`, we will replace the base url with s3-bucket base-url. So, in this way the final request will be made to the actual url i.e.
+https://c90081bas2001edx.s3.amazonaws.com/scorm/503a49b7ed1d4a2caa22af84df87fa8c/f792c4c021da74aac780e072bf16aa0ed4987767/shared/launchpage.html
+
 
 License
 -------
 
-This work is licensed under the terms of the `GNU Affero General Public License (AGPL) <https://github.com/overhangio/openedx-scorm-xblock/blob/master/LICENSE.txt>`_.
+This work is licensed under the terms of the `GNU Affero General Public License (AGPL) <https://github.com/overhangio/openedx-scorm-xblock/blob/master/LICENSE.txt>`_.

openedxscorm/scormxblock.py

@@ -1,19 +1,28 @@
+from urllib.parse import urlparse
+import importlib
 import json
 import hashlib
 import os
+import tempfile
 import logging
 import re
 import xml.etree.ElementTree as ET
 import zipfile
+import concurrent.futures
+import requests
+import mimetypes
+import boto3
 
+from django.conf import settings
+from completion import waffle as completion_waffle
 from django.core.files import File
-from django.core.files.storage import default_storage
 from django.template import Context, Template
 from django.utils import timezone
 from webob import Response
 import pkg_resources
 
 from web_fragments.fragment import Fragment
+from xblock.completable import XBlockCompletionMode
 from xblock.core import XBlock
 from xblock.fields import Scope, String, Float, Boolean, Dict, DateTime, Integer
 
@@ -24,26 +33,35 @@ def _(text):
 
 logger = logging.getLogger(__name__)
 
+# importing directly from settings.XBLOCK_SETTINGS doesn't work here... doesn't have vals from ENV TOKENS yet
+scorm_settings = settings.ENV_TOKENS["XBLOCK_SETTINGS"]["ScormXBlock"]
+SCORM_FILE_STORAGE_TYPE = scorm_settings.get("SCORM_FILE_STORAGE_TYPE", "django.core.files.storage.default_storage")
+SCORM_MEDIA_BASE_URL = scorm_settings.get("SCORM_MEDIA_BASE_URL", "/scorm")
+mod, store_class = SCORM_FILE_STORAGE_TYPE.rsplit(".", 1)
+scorm_storage_module = importlib.import_module(mod)
+scorm_storage_class = getattr(scorm_storage_module, store_class)
+if SCORM_FILE_STORAGE_TYPE.endswith("default_storage"):
+    scorm_storage_instance = scorm_storage_class
+else:
+    scorm_storage_instance = scorm_storage_class()
+
 
 @XBlock.wants("settings")
 class ScormXBlock(XBlock):
     """
     When a user uploads a Scorm package, the zip file is stored in:
-
         media/{org}/{course}/{block_type}/{block_id}/{sha1}{ext}
-
     This zip file is then extracted to the media/{scorm_location}/{block_id}.
-
     The scorm location is defined by the LOCATION xblock setting. If undefined, this is
     "scorm". This setting can be set e.g:
-
         XBLOCK_SETTINGS["ScormXBlock"] = {
             "LOCATION": "alternatevalue",
         }
-
     Note that neither the folder the folder nor the package file are deleted when the
     xblock is removed.
     """
+    has_custom_completion = True
+    completion_mode = XBlockCompletionMode.COMPLETABLE
 
     display_name = String(
         display_name=_("Display Name"),
@@ -156,6 +174,70 @@ def json_response(data):
             json.dumps(data), content_type="application/json", charset="utf8"
         )
 
+    @XBlock.handler
+    def scorm_view(self, request, _suffix):
+        """
+        View for serving SCORM content. It receives a request with the path to the SCORM content to serve, generates a pre-signed
+        URL to access the content in the AWS S3 bucket, retrieves the file content and returns it with the appropriate content
+        type.
+
+        Parameters:
+        ----------
+        request : django.http.request.HttpRequest
+            HTTP request object containing the path to the SCORM content to serve.
+        _suffix : str
+            Unused parameter.
+
+        Returns:
+        -------
+        Response object containing the content of the requested file with the appropriate content type.
+        """
+        path = request.url.split('scorm_view/')[-1]
+        path = re.sub(r"(\?.*|:[\d:]*$)", "", path)
+        file_name = os.path.basename(path)
+        signed_url = self.get_presigned_url(path)
+        if not signed_url:
+            return signed_url
+
+        file_content = requests.get(signed_url).content
+        file_type, _ = mimetypes.guess_type(file_name)
+
+        return Response(
+            file_content, content_type=file_type
+        )
+
+
+    def get_presigned_url(self, file_path):
+        """
+        Generates a pre-signed URL to access a file in an AWS S3 bucket.
+
+        Parameters:
+        ----------
+        file_path : str
+            The path to the file to access in the S3 bucket.
+
+        Returns:
+        -------
+        str
+            The pre-signed URL for accessing the file.
+        """
+        presigned_url = ""
+        if self.storage.exists(os.path.join(self.extract_folder_path, 
+                                            file_path)):
+            expires_in = 86400
+            # Get a boto3 client instance for S3
+            s3_client = boto3.client('s3',
+                                    aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
+                                    aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
+                                    region_name=settings.AWS_S3_REGION_NAME)
+            # Generate the presigned URL
+            presigned_url = s3_client.generate_presigned_url('get_object',
+                                                            Params={'Bucket': settings.AWS_STORAGE_BUCKET_NAME,
+                                                                    'Key': os.path.join(self.extract_folder_path, file_path)},
+                                                            ExpiresIn=expires_in)
+
+        return presigned_url
+
     @XBlock.handler
     def studio_submit(self, request, _suffix):
         self.display_name = request.params["display_name"]
@@ -174,29 +256,50 @@ def studio_submit(self, request, _suffix):
         self.update_package_meta(package_file)
 
         # First, save scorm file in the storage for mobile clients
-        if default_storage.exists(self.package_path):
+        if scorm_storage_instance.exists(self.package_path):
             logger.info('Removing previously uploaded "%s"', self.package_path)
-            default_storage.delete(self.package_path)
-        default_storage.save(self.package_path, File(package_file))
+            scorm_storage_instance.delete(self.package_path)
+        scorm_storage_instance.save(self.package_path, File(package_file))
         logger.info('Scorm "%s" file stored at "%s"', package_file, self.package_path)
 
         # Then, extract zip file
-        if default_storage.exists(self.extract_folder_base_path):
+        if scorm_storage_instance.exists(self.extract_folder_base_path):
             logger.info(
                 'Removing previously unzipped "%s"', self.extract_folder_base_path
             )
             recursive_delete(self.extract_folder_base_path)
+
+        def unzip_member(_scorm_storage_instance,uncompressed_file,extract_folder_path, filename):
+            logger.info('Started uploading file {fname}'.format(fname=filename))
+            _scorm_storage_instance.save(
+                os.path.join(extract_folder_path, filename),
+                uncompressed_file,
+            )
+            logger.info('End uploadubg file {fname}'.format(fname=filename))
+
         with zipfile.ZipFile(package_file, "r") as scorm_zipfile:
-            for zipinfo in scorm_zipfile.infolist():
-                # Do not unzip folders, only files. In Python 3.6 we will have access to
-                # the is_dir() method to verify whether a ZipInfo object points to a
-                # directory.
-                # https://docs.python.org/3.6/library/zipfile.html#zipfile.ZipInfo.is_dir
-                if not zipinfo.filename.endswith("/"):
-                    default_storage.save(
-                        os.path.join(self.extract_folder_path, zipinfo.filename),
-                        scorm_zipfile.open(zipinfo.filename),
-                    )
+            futures = []
+            with concurrent.futures.ThreadPoolExecutor() as executor:
+                logger.info("started concurrent.futures.ThreadPoolExecutor")
+                for zipinfo in scorm_zipfile.infolist():
+                    fp = tempfile.TemporaryFile()
+                    fp.write(scorm_zipfile.open(zipinfo.filename).read())
+                    logger.info("started uploadig file {fname}".format(fname=zipinfo.filename))
+                    # Do not unzip folders, only files. In Python 3.6 we will have access to
+                    # the is_dir() method to verify whether a ZipInfo object points to a
+                    # directory.
+                    # https://docs.python.org/3.6/library/zipfile.html#zipfile.ZipInfo.is_dir
+                    if not zipinfo.filename.endswith("/"):
+                        futures.append(
+                            executor.submit(
+                                unzip_member,
+                                scorm_storage_instance,
+                                fp,
+                                self.extract_folder_path,
+                                zipinfo.filename,
+                            )
+                        )
+                logger.info("end concurrent.futures.ThreadPoolExecutor")
 
         try:
             self.update_package_fields()
@@ -209,15 +312,8 @@ def studio_submit(self, request, _suffix):
     def index_page_url(self):
         if not self.package_meta or not self.index_page_path:
             return ""
-        folder = self.extract_folder_path
-        if default_storage.exists(
-            os.path.join(self.extract_folder_base_path, self.index_page_path)
-        ):
-            # For backward-compatibility, we must handle the case when the xblock data
-            # is stored in the base folder.
-            folder = self.extract_folder_base_path
-            logger.warning("Serving SCORM content from old-style path: %s", folder)
-        return default_storage.url(os.path.join(folder, self.index_page_path))
+        url = settings.CMS_BASE + "/xblock/" + str(self.scope_ids.usage_id) + "/handler/scorm_view/" + self.index_page_path
+        return url
 
     @property
     def package_path(self):
@@ -293,6 +389,35 @@ def publish_grade(self):
         self.runtime.publish(
             self, "grade", {"value": self.get_grade(), "max_value": self.weight},
         )
+        self.publish_completion()
+
+
+    def publish_completion(self):
+        """
+        Mark scorm xbloxk as completed if user has completed the scorm course unit.
+
+        it will work along with the edX completion tool: https://github.com/edx/completion
+        """
+        if not completion_waffle.waffle().is_enabled(completion_waffle.ENABLE_COMPLETION_TRACKING):
+            return
+
+        if XBlockCompletionMode.get_mode(self) != XBlockCompletionMode.COMPLETABLE:
+            return
+
+        completion_value = 0.0
+        if not self.has_score:
+            # component does not have any score
+            if self.get_completion_status() == "completed":
+                completion_value = 1.0
+        else:
+            if self.get_completion_status() in ["passed", "failed"]:
+                completion_value = 1.0
+
+        data = {
+            "completion": completion_value
+        }
+        self.runtime.publish(self, "completion", data)
+
 
     def get_grade(self):
         lesson_score = self.lesson_score
@@ -331,7 +456,7 @@ def update_package_fields(self):
         self.index_page_path = ""
         imsmanifest_path = os.path.join(self.extract_folder_path, "imsmanifest.xml")
         try:
-            imsmanifest_file = default_storage.open(imsmanifest_path)
+            imsmanifest_file = scorm_storage_instance.open(imsmanifest_path)
         except IOError:
             raise ScormError(
                 "Invalid package: could not find 'imsmanifest.xml' file at the root of the zip file"
@@ -408,7 +533,7 @@ def student_view_data(self):
         if self.index_page_url:
             return {
                 "last_modified": self.package_meta.get("last_updated", ""),
-                "scorm_data": default_storage.url(self.package_path),
+                "scorm_data": scorm_storage_instance.url(self.package_path),
                 "size": self.package_meta.get("size", 0),
                 "index_page": self.index_page_path,
             }
@@ -434,11 +559,11 @@ def recursive_delete(root):
     Unfortunately, this will not delete empty folders, as the default FileSystemStorage
     implementation does not allow it.
     """
-    directories, files = default_storage.listdir(root)
+    directories, files = scorm_storage_instance.listdir(root)
     for directory in directories:
         recursive_delete(os.path.join(root, directory))
     for f in files:
-        default_storage.delete(os.path.join(root, f))
+        scorm_storage_instance.delete(os.path.join(root, f))
 
 
 class ScormError(Exception):