Merge pull request #59 from dcppc/hotfix/column-width

hot fix - specify column width

.gitignore

@@ -1,4 +1,5 @@
-config_*
+config_centillion.py
+config_flask.py
 vp
 credentials.json
 drive*.json

.gitmodules

@@ -1,3 +1,3 @@
-[submodule "mkdocs-material"]
-	path = mkdocs-material
-	url = https://git.charlesreid1.com/charlesreid1/mkdocs-material.git
+[submodule "mkdocs-material-dib"]
+	path = mkdocs-material-dib
+	url = https://github.com/dib-lab/mkdocs-material-dib.git

LICENSE

@@ -1,29 +1,133 @@
-BSD 3-Clause License
+# DEDICATED TO THE PUBLIC DOMAIN
 
-Copyright (c) 2018, Charles Reid
-All rights reserved.
+The dcppc/organize repository has been dedicated to the public domain.
+It is protected by the Creative Commons CC0 Universal Public Domain
+Dedication license. You can read the entire license below or at
+<http://creativecommons.org/publicdomain/zero/1.0/deed.en>.
 
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
+# CC0 UNIVERSAL PUBLIC DOMAIN DEDICATION LICENSE
 
-* Redistributions of source code must retain the above copyright notice, this
-  list of conditions and the following disclaimer.
+## Statement of Purpose
 
-* Redistributions in binary form must reproduce the above copyright notice,
-  this list of conditions and the following disclaimer in the documentation
-  and/or other materials provided with the distribution.
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator
+and subsequent owner(s) (each and all, an "owner") of an original work
+of authorship and/or a database (each, a "Work").
 
-* Neither the name of the copyright holder nor the names of its
-  contributors may be used to endorse or promote products derived from
-  this software without specific prior written permission.
+Certain owners wish to permanently relinquish those rights to a Work for
+the purpose of contributing to a commons of creative, cultural and
+scientific works ("Commons") that the public can reliably and without
+fear of later claims of infringement build upon, modify, incorporate in
+other works, reuse and redistribute as freely as possible in any form
+whatsoever and for any purposes, including without limitation commercial
+purposes. These owners may contribute to the Commons to promote the
+ideal of a free culture and the further production of creative, cultural
+and scientific works, or to gain reputation or greater distribution for
+their Work in part through the use and efforts of others.
 
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
-FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
-SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
-CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
-OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+For these and/or other purposes and motivations, and without any
+expectation of additional consideration or compensation, the person
+associating CC0 with a Work (the "Affirmer"), to the extent that he or
+she is an owner of Copyright and Related Rights in the Work, voluntarily
+elects to apply CC0 to the Work and publicly distribute the Work under
+its terms, with knowledge of his or her Copyright and Related Rights in
+the Work and the meaning and intended legal effect of CC0 on those
+rights.
+
+1.  **Copyright and Related Rights.** A Work made available under CC0
+    may be protected by copyright and related or neighboring rights
+    ("Copyright and Related Rights"). Copyright and Related Rights
+    include, but are not limited to, the following:
+    
+    i. the right to reproduce, adapt, distribute, perform, display,
+    communicate, and translate a Work;
+    
+    ii. moral rights retained by the original author(s) and/or
+    performer(s);
+    
+    iii. publicity and privacy rights pertaining to a person's image or
+    likeness depicted in a Work;
+    
+    iv. rights protecting against unfair competition in regards to a
+    Work, subject to the limitations in paragraph 4(a), below;
+    
+    v. rights protecting the extraction, dissemination, use and reuse of
+    data in a Work;
+    
+    vi. database rights (such as those arising under Directive 96/9/EC
+    of the European Parliament and of the Council of 11 March 1996 on
+    the legal protection of databases, and under any national
+    implementation thereof, including any amended or successor version
+    of such directive); and
+    
+    vii. other similar, equivalent or corresponding rights throughout
+    the world based on applicable law or treaty, and any national
+    implementations thereof.
+
+2.  **Waiver.** To the greatest extent permitted by, but not in
+    contravention of, applicable law, Affirmer hereby overtly, fully,
+    permanently, irrevocably and unconditionally waives, abandons, and
+    surrenders all of Affirmer's Copyright and Related Rights and
+    associated claims and causes of action, whether now known or unknown
+    (including existing as well as future claims and causes of action),
+    in the Work (i) in all territories worldwide, (ii) for the maximum
+    duration provided by applicable law or treaty (including future time
+    extensions), (iii) in any current or future medium and for any
+    number of copies, and (iv) for any purpose whatsoever, including
+    without limitation commercial, advertising or promotional purposes
+    (the "Waiver"). Affirmer makes the Waiver for the benefit of each
+    member of the public at large and to the detriment of Affirmer's
+    heirs and successors, fully intending that such Waiver shall not be
+    subject to revocation, rescission, cancellation, termination, or any
+    other legal or equitable action to disrupt the quiet enjoyment of
+    the Work by the public as contemplated by Affirmer's express
+    Statement of Purpose.
+
+3.  **Public License Fallback.** Should any part of the Waiver for any
+    reason be judged legally invalid or ineffective under applicable
+    law, then the Waiver shall be preserved to the maximum extent
+    permitted taking into account Affirmer's express Statement of
+    Purpose. In addition, to the extent the Waiver is so judged Affirmer
+    hereby grants to each affected person a royalty-free, non
+    transferable, non sublicensable, non exclusive, irrevocable and
+    unconditional license to exercise Affirmer's Copyright and Related
+    Rights in the Work (i) in all territories worldwide, (ii) for the
+    maximum duration provided by applicable law or treaty (including
+    future time extensions), (iii) in any current or future medium and
+    for any number of copies, and (iv) for any purpose whatsoever,
+    including without limitation commercial, advertising or promotional
+    purposes (the "License"). The License shall be deemed effective as
+    of the date CC0 was applied by Affirmer to the Work. Should any part
+    of the License for any reason be judged legally invalid or
+    ineffective under applicable law, such partial invalidity or
+    ineffectiveness shall not invalidate the remainder of the License,
+    and in such case Affirmer hereby affirms that he or she will not (i)
+    exercise any of his or her remaining Copyright and Related Rights in
+    the Work or (ii) assert any associated claims and causes of action
+    with respect to the Work, in either case contrary to Affirmer's
+    express Statement of Purpose.
+
+4.  **Limitations and Disclaimers.**
+    
+    a. No trademark or patent rights held by Affirmer are waived,
+    abandoned, surrendered, licensed or otherwise affected by this
+    document.
+    
+    b. Affirmer offers the Work as-is and makes no representations or
+    warranties of any kind concerning the Work, express, implied,
+    statutory or otherwise, including without limitation warranties of
+    title, merchantability, fitness for a particular purpose, non
+    infringement, or the absence of latent or other defects, accuracy,
+    or the present or absence of errors, whether or not discoverable,
+    all to the greatest extent permissible under applicable law.
+    
+    c. Affirmer disclaims responsibility for clearing rights of other
+    persons that may apply to the Work or any use thereof, including
+    without limitation any person's Copyright and Related Rights in the
+    Work. Further, Affirmer disclaims responsibility for obtaining any
+    necessary consents, permissions or other rights required for any use
+    of the Work.
+    
+    d. Affirmer understands and acknowledges that Creative Commons is
+    not a party to this document and has no duty or obligation with
+    respect to this CC0 or use of the Work.

Readme.md

@@ -1,18 +1,19 @@
-# The Centillion
+# Centillion
 
-**the centillion**: a pan-github-markdown-issues-google-docs search engine.
+**centillion**: a pan-github-markdown-issues-google-docs search engine.
 
 **a centillion**: a very large number consisting of a 1 with 303 zeros after it.
 
-the centillion is 3.03 log-times better than the googol.
+one centillion is 3.03 log-times better than a googol.
 
-![Screen shot of centillion](img/ss.png)
+![Screen shot: centillion search](docs/images/search.png)
 
 
-## what is it
+## What Is It
 
-The centillion is a search engine built using [whoosh](https://whoosh.readthedocs.io/en/latest/intro.html),
-a Python library for building search engines.
+Centillion (https://github.com/dcppc/centillion) is a search engine that can index 
+three kinds of collections: Google Documents (.docx files), Github issues, and Markdown files in 
+Github repos.
 
 We define the types of documents the centillion should index,
 what info and how. The centillion then builds and
@@ -24,13 +25,113 @@ defined in `centillion.py`.
 
 The centillion keeps it simple.
 
+## Authentication Layer
 
-## quickstart (with Github auth)
+Centillion lives behind a Github authentication layer, implemented with 
+[flask-dance](https://github.com/singingwolfboy/flask-dance). When you first
+visit the site it will ask you to authenticate with Github so that it can 
+verify you have permission to access the site.
+
+![Screen shot: centillion authentication](docs/images/auth.png)
+
+## Master List
+
+There is a master list of all content indexed by centilion at the master list page,
+<https://search.nihdatacommons.us/master_list>.
+
+A master list for each type of document indexed by the search engine is displayed
+in a table:
+
+![Screen shot: centillion master list](docs/images/master_list.png)
+
+The metadata shown in these tables can be filtered and sorted:
+
+![Screen shot: centillion master list with sorting](docs/images/master_list2.png)
+
+## Control Panel
+
+There's also a control panel at <https://search.nihdatacommons.us/control_panel> 
+that allows you to rebuild the search index from scratch.  The search index
+stores versions/contents of files locally, so re-indexing involves going out and
+asking each API for new versions of a file/document/web page. When you re-index
+the main search index, it will ask every API for new versions of every document.
+You can also update only specific types of documents in the search index.
+
+![Screen shot: centillion control panel](docs/images/control_panel.png)
+
+
+
+## Technologies
+
+Centillion is a Python program built using whoosh (search engine library). It
+indexes the full text of docx files in Google Documents, just the filenames for
+non-docx files. The full text of issues and their comments are indexed, and
+results are grouped by issue. Centillion requires Google Drive and Github OAuth
+apps. Once you provide credentials to Flask you're all set to go. 
+
+
+## Configuration
+
+You will need to configure both the centillion search index and the flask app.
+
+The centillion search index is configured with `config_centillion.py`; this file
+sets the names of repositories to crawl when indxing issues and files.
+
+The flask app is configured with `config_flask.py`. This file contains sensitive
+information and is in the `.gitignore` file. This file contains API credentials 
+for Github and Groups.io.
+
+Exampls are provided in `config_centillion.example.py` and `config_flask.example.py`.
+
+
+## Authentication
+
+The search engine will need to connect to several APIs when it re-indexes the
+search index:
+
+* Github
+* Groups.io
+* Google Drive
+
+### Github
+
+Github API credentials (both an OAuth token for the centillion app's Github
+authentication mechanism, and a personal access token for accessing repositories
+during the re-indexing process) are provided in `config_flask.py`.
+
+### Groups.io
+
+The Groups.io API token is used to index email threads. This token is provided in
+`config_flask.py`.
+
+### Google Drive
+
+The Google Drive API credentials are provided in a file, `credentials.json`. This is
+the file that is generated when the OAuth process is complete.
+
+When you enable the Google Drive API in the Google Cloud Console, you will be provided
+with a file `client_secrets.json`. To authenticate centillion with Google Drive, you should
+download this file, and run the Google Drive utility directly:
+
+```
+python gdrive_util.py
+```
+
+This will initiate the authentication procedure. Sign in as a user that has access to
+the documents you want to index, and _only_ the documents you want to index (it is useful
+to set up a bot account for this purpose).
+
+Once you log in as that user, it will create `credentials.json`, and the Google Drive
+re-indexing procedure should not have any problems autheticating using that file.
+
+## Quickstart (With Github Auth)
 
 Start by creating a Github OAuth application.
 Get the public and private application key 
 (client token and client secret token)
 from the Github application's page.
+You will also need a Github access token
+(in addition to the app tokens).
 
 When you create the application, set the callback
 URL to `/login/github/authorized`, as in:
@@ -58,18 +159,10 @@ This will start a Flask server, and you can view the minimal search engine
 interface in your browser at `http://<ip>:5000`.
 
 
-## troubleshooting
+## Troubleshooting
 
 If you are having problems with your callback URL being treated
 as HTTP by Github, even though there is an HTTPS address, and
 everything else seems fine, try deleting the Github OAuth app
 and creating a new one.
 
-
-## more info
-
-For more info see the documentation: <https://charlesreid1.github.io/centillion>
-
-
-
-

centillion.py

@@ -1,16 +1,17 @@
 import threading
-from subprocess import call
+import subprocess
 
 import codecs
 import os, json
 
 from werkzeug.contrib.fixers import ProxyFix
-from flask import Flask, request, redirect, url_for, render_template, flash
+from flask import Flask, request, redirect, url_for, render_template, flash, jsonify
 from flask_dance.contrib.github import make_github_blueprint, github
 
 # create our application
 from centillion_search import Search
 
+import config_centillion
 
 """
 The Centillion
@@ -27,10 +28,17 @@ You provide:
 
 
 class UpdateIndexTask(object):
-    def __init__(self, gh_oauth_token, diff_index=False):
+    def __init__(self, app_config, diff_index=False,run_which='all'):
         self.diff_index = diff_index
+        self.run_which = run_which
         thread = threading.Thread(target=self.run, args=())
-        self.gh_oauth_token = gh_oauth_token
+
+        self.gh_token = app_config['GITHUB_TOKEN']
+        self.groupsio_credentials = {
+                'groupsio_token' :     app_config['GROUPSIO_TOKEN'],
+                'groupsio_username' :  app_config['GROUPSIO_USERNAME'],
+                'groupsio_password' :  app_config['GROUPSIO_PASSWORD']
+        }
         thread.daemon = True
         thread.start()
 
@@ -40,13 +48,13 @@ class UpdateIndexTask(object):
         if(self.diff_index):
             raise Exception("diff index not implemented")
 
-        from get_centillion_config import get_centillion_config
-        config = get_centillion_config('config_centillion.json')
-
-        search.update_index_markdown(self.gh_oauth_token,config)
-        search.update_index_issues(self.gh_oauth_token,config)
-        search.update_index_gdocs(config)
+        #from get_centillion_config import get_centillion_config
+        config = config_centillion.config
 
+        search.update_index(self.groupsio_credentials,
+                            self.gh_token,
+                            self.run_which,
+                            config)
 
 
 app = Flask(__name__)
@@ -55,166 +63,239 @@ app.wsgi_app = ProxyFix(app.wsgi_app)
 # Load default config and override config from an environment variable
 app.config.from_pyfile("config_flask.py")
 
-github_bp = make_github_blueprint()
-#github_bp = make_github_blueprint(
-#                        client_id = os.environ.get('GITHUB_OAUTH_CLIENT_ID'),
-#                        client_secret = os.environ.get('GITHUB_OAUTH_CLIENT_SECRET'),
-#                        scope='read:org')
+#github_bp = make_github_blueprint()
+github_bp = make_github_blueprint(
+                        client_id = os.environ.get('GITHUB_OAUTH_CLIENT_ID'),
+                        client_secret = os.environ.get('GITHUB_OAUTH_CLIENT_SECRET'),
+                        scope='read:org')
 
 app.register_blueprint(github_bp, url_prefix="/login")
 
-contents404 = "<html><body><h1>Status: Error 404 Page Not Found</h1></body></html>"
-contents403 = "<html><body><h1>Status: Error 403 Access Denied</h1></body></html>"
-contents200 = "<html><body><h1>Status: OK 200</h1></body></html>"
-
+last_searches_file = app.config["INDEX_DIR"] + "/last_searches.txt"
 
 ##############################
 # Flask routes
 
 @app.route('/')
 def index():
-
     if not github.authorized:
-        return redirect(url_for("github.login"))
-
+        return render_template("landing.html")
     else:
-
         username = github.get("/user").json()['login']
-
         resp = github.get("/user/orgs")
         if resp.ok:
 
-            # If they are in team copper, redirect to search.
+            # If they are in dcppc, redirect to search.
             # Otherwise, hit em with a 403
             all_orgs = resp.json()
             for org in all_orgs:
                 if org['login']=='dcppc':
-                    copper_team_id = '2700235'
-                    mresp = github.get('/teams/%s/members/%s'%(copper_team_id,username))
-                    if mresp.status_code==204:
+                    # Business as usual
+                    return redirect(url_for("search", query="", fields=""))
 
-                        # --------------------
-                        # Business as usual
-                        return redirect(url_for("search", query="", fields=""))
+            # Not in dcppc
+            return render_template('403.html')
 
-            return contents403
+        # Could not reach Github
+        return render_template('404.html')
+
+
+@app.route('/log_in')
+def log_in():
+    if not github.authorized:
+        return redirect(url_for("github.login"))
+    else:
+        username = github.get("/user").json()['login']
+        resp = github.get("/user/orgs")
+        if resp.ok:
+
+            # If they are in dcppc, redirect to search.
+            # Otherwise, hit em with a 403
+            all_orgs = resp.json()
+            for org in all_orgs:
+                if org['login']=='dcppc':
+                    # Business as usual
+                    return redirect(url_for("search", query="", fields=""))
+
+            # Not in dcppc
+            return render_template('403.html')
+
+        # Could not reach Github
+        return render_template('404.html')
 
-        return contents404
 
-### @app.route('/')
-### def index():
-###     return redirect(url_for("search", query="", fields=""))
 
 @app.route('/search')
 def search():
-
     if not github.authorized:
         return redirect(url_for("github.login"))
-
     username = github.get("/user").json()['login']
-
     resp = github.get("/user/orgs")
     if resp.ok:
 
+        # If they are in dcppc, show them search.html
+        # Otherwise, hit em with a 403
         all_orgs = resp.json()
         for org in all_orgs:
             if org['login']=='dcppc':
+                # Business as usual
+                query = request.args['query']
+                fields = request.args.get('fields')
+                if fields == 'None':
+                    fields = None
 
-                copper_team_id = '2700235'
+                search = Search(app.config["INDEX_DIR"])
+                if not query:
+                    parsed_query = ""
+                    result = []
 
-                mresp = github.get('/teams/%s/members/%s'%(copper_team_id,username))
-                if mresp.status_code==204:
+                else:
+                    parsed_query, result = search.search(query.split(), fields=[fields])
+                    store_search(query,fields)
 
-                    # --------------------
-                    # Business as usual
-                    query = request.args['query']
-                    fields = request.args.get('fields')
-                    if fields == 'None':
-                        fields = None
+                totals = search.get_document_total_count()
 
-                    search = Search(app.config["INDEX_DIR"])
-                    if not query:
-                        parsed_query = ""
-                        result = []
+                return render_template('search.html', 
+                                       entries=result, 
+                                       query=query, 
+                                       parsed_query=parsed_query, 
+                                       fields=fields, 
+                                       totals=totals)
 
-                    else:
-                        parsed_query, result = search.search(query.split(), fields=[fields])
+        # Not in dcppc 
+        return render_template('403.html')
 
-                    totals = search.get_document_total_count()
-
-                    return render_template('search.html', 
-                                           entries=result, 
-                                           query=query, 
-                                           parsed_query=parsed_query, 
-                                           fields=fields, 
-                                           totals=totals)
-
-    return contents403
+    # Could not reach Github
+    return render_template('404.html')
 
 
-@app.route('/update_index')
-def update_index():
-
+@app.route('/update_index/<run_which>')
+def update_index(run_which):
+    """
+    TEAM COPPER ONLY!!!
+    """
     if not github.authorized:
         return redirect(url_for("github.login"))
-
     username = github.get("/user").json()['login']
-
     resp = github.get("/user/orgs")
     if resp.ok:
 
+        # Only Team Copper members can update the index
         all_orgs = resp.json()
         for org in all_orgs:
             if org['login']=='dcppc':
-
                 copper_team_id = '2700235'
-
                 mresp = github.get('/teams/%s/members/%s'%(copper_team_id,username))
                 if mresp.status_code==204:
-
-                    gh_oauth_token = github.token['access_token']
-
-                    # --------------------
                     # Business as usual
-                    UpdateIndexTask(gh_oauth_token, diff_index=False)
+                    UpdateIndexTask(app.config,
+                                    diff_index=False,
+                                    run_which = run_which)
                     flash("Rebuilding index, check console output")
-                    return render_template("controlpanel.html", 
-                                           totals={})
-
-    return contents403
+                    # This redirects user to /control_panel route
+                    # to prevent accidental re-indexing
+                    return redirect(url_for("control_panel"))
 
+    return render_template('403.html')
 
 
 @app.route('/control_panel')
 def control_panel():
-
+    """
+    TEAM COPPER ONLY!!!
+    """
     if not github.authorized:
         return redirect(url_for("github.login"))
-
     username = github.get("/user").json()['login']
-
     resp = github.get("/user/orgs")
     if resp.ok:
 
+        # Only Team Copper members can access the control panel
         all_orgs = resp.json()
         for org in all_orgs:
             if org['login']=='dcppc':
-
                 copper_team_id = '2700235'
-
                 mresp = github.get('/teams/%s/members/%s'%(copper_team_id,username))
                 if mresp.status_code==204:
+                    # Business as usual
+                    return render_template("controlpanel.html")
 
-                    return render_template("controlpanel.html", 
-                                           totals={})
+        # Not in dcppc 
+        return render_template('403.html')
 
-    return contents403
+    # Could not reach Github
+    return render_template('404.html')
 
 
+
+@app.route('/master_list')
+def master_list():
+    if not github.authorized:
+        return redirect(url_for("github.login"))
+    username = github.get("/user").json()['login']
+    resp = github.get("/user/orgs")
+    if resp.ok:
+
+        # If they are in dcppc, show them masterlist.html
+        # Otherwise, hit em with a 403
+        all_orgs = resp.json()
+        for org in all_orgs:
+            if org['login']=='dcppc':
+                # Business as usual
+                return render_template("masterlist.html")
+
+        # Not in dcppc 
+        return render_template('403.html')
+
+    # Could not reach Github
+    return render_template('404.html')
+
+
+
+@app.route('/list/<doctype>')
+def list_docs(doctype):
+    if not github.authorized:
+        return redirect(url_for("github.login"))
+    username = github.get("/user").json()['login']
+    resp = github.get("/user/orgs")
+    if resp.ok:
+        all_orgs = resp.json()
+        for org in all_orgs:
+            if org['login']=='dcppc':
+                copper_team_id = '2700235'
+                mresp = github.get('/teams/%s/members/%s'%(copper_team_id,username))
+                if mresp.status_code==204:
+                    # Business as usual
+                    search = Search(app.config["INDEX_DIR"])
+                    return jsonify(search.get_list(doctype))
+
+    return render_template('403.html')
+
 @app.errorhandler(404)
 def oops(e):
-    return contents404
+    return render_template('404.html')
+
+
+def store_search(query, fields):
+    """
+    Store searches in a text file
+    """
+    if os.path.exists(last_searches_file):
+        with codecs.open(last_searches_file, 'r', encoding='utf-8') as f:
+            contents = f.readlines()
+    else:
+        contents = []
+
+    search = "query=%s&fields=%s\n" % (query, fields)
+    if not search in contents:
+        contents.insert(0, search)
+
+    with codecs.open(last_searches_file, 'w', encoding='utf-8') as f:
+        f.writelines(contents)
+
 
 if __name__ == '__main__':
+    # if running local instance, set to true
+    os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = 'true'
     app.run(host="0.0.0.0",port=5000)
 

centillion_search.py

@@ -1,10 +1,11 @@
 import shutil
 import html.parser
 
-from github import Github
+from github import Github, GithubException
 import base64
 
 from gdrive_util import GDrive
+from groupsio_util import GroupsIOArchivesCrawler, GroupsIOException
 from apiclient.http import MediaIoBaseDownload
 
 import mistune
@@ -16,6 +17,7 @@ import pypandoc
 import os.path
 import codecs
 from datetime import datetime
+import dateutil.parser
 
 from whoosh.qparser import MultifieldParser, QueryParser
 from whoosh.analysis import StemmingAnalyzer
@@ -98,6 +100,58 @@ class Search:
         self.open_index(index_folder)
 
 
+    # ------------------------------
+    # Update the entire index
+
+    def update_index(self, groupsio_credentials, gh_token, run_which, config):
+        """
+        Update the entire search index
+        """
+        if run_which=='all' or run_which=='emailthreads':
+            try:
+                self.update_index_emailthreads(groupsio_credentials, config)
+            except GroupsIOException as e:
+                print("ERROR: While re-indexing: failed to update Groups.io email threads, hit API rate limit")
+                print("-"*40)
+                print(repr(e))
+                print("-"*40)
+                print("Continuing...")
+                pass
+
+        if run_which=='all' or run_which=='ghfiles':
+            try:
+                self.update_index_ghfiles(gh_token,config)
+            except Exception as e:
+                print("ERROR: While re-indexing: failed to update Github files")
+                print("-"*40)
+                print(repr(e))
+                print("-"*40)
+                print("Continuing...")
+                pass
+
+        if run_which=='all' or run_which=='issues':
+            try:
+                self.update_index_issues(gh_token,config)
+            except Exception as e:
+                print("ERROR: While re-indexing: failed to update Github issues")
+                print("-"*40)
+                print(repr(e))
+                print("-"*40)
+                print("Continuing...")
+                pass
+
+        if run_which=='all' or run_which=='gdocs':
+            try:
+                self.update_index_gdocs(config)
+            except Exception as e:
+                print("ERROR: While re-indexing: failed to update Google Drive files")
+                print("-"*40)
+                print(repr(e))
+                print("-"*40)
+                print("Continuing...")
+                pass
+
+
     # ------------------------------
     # Create a schema and open a search index
     # on disk.
@@ -128,7 +182,6 @@ class Search:
         schema = Schema(
                 id = ID(stored=True, unique=True),
                 kind = ID(stored=True),
-                #fingerprint = ID(stored=True),
 
                 created_time = ID(stored=True),
                 modified_time = ID(stored=True),
@@ -252,7 +305,6 @@ class Search:
             with open(fullpath_input, 'wb') as f:
                 f.write(r.content)
 
-
             # Try to convert docx file to plain text
             try:
                 output = pypandoc.convert_file(fullpath_input,
@@ -267,7 +319,6 @@ class Search:
 
             # If export was successful, read contents of markdown
             # into the content variable.
-            # into the content variable.
             if os.path.isfile(fullpath_output):
                 # Export was successful
                 with codecs.open(fullpath_output, encoding='utf-8') as f:
@@ -277,12 +328,14 @@ class Search:
             # No matter what happens, clean up.
             print(" > Cleaning up \"%s\""%item['name'])
 
-            subprocess.call(['rm','-fr',fullpath_output])
+            ## test
             #print(" ".join(['rm','-fr',fullpath_output]))
-
-            subprocess.call(['rm','-fr',fullpath_input])
             #print(" ".join(['rm','-fr',fullpath_input]))
 
+            # do it
+            subprocess.call(['rm','-fr',fullpath_output])
+            subprocess.call(['rm','-fr',fullpath_input])
+
             if update:
                 print(" > Removing old record")
                 writer.delete_by_term('id',item['id'])
@@ -316,7 +369,7 @@ class Search:
     # to a search index.
 
 
-    def add_issue(self, writer, issue, config, update=True):
+    def add_issue(self, writer, issue, gh_token, config, update=True):
         """
         Add a Github issue/comment to a search index.
         """
@@ -368,69 +421,150 @@ class Search:
 
 
 
-    def add_markdown(self, writer, d, config, update=True):
+    # ------------------------------
+    # Add a single github file 
+    # to a search index.
+
+    def add_ghfile(self, writer, d, gh_token, config, update=True):
         """
-        Use a Github markdown document API record
-        to add a markdown document's contents to
-        the search index.
+        Use a Github file API record to add a filename
+        to the search index.
         """
+        MARKDOWN_EXTS = ['.md','.markdown']
+
         repo = d['repo']
         org = d['org']
         repo_name = org + "/" + repo
         repo_url = "https://github.com/" + repo_name
 
-        fpath = d['path']
-        furl = d['url']
-        fsha = d['sha']
-        _, fname = os.path.split(fpath)
-        _, fext = os.path.splitext(fpath)
-
-        print("Indexing markdown doc %s"%(fname))
-
-        # Unpack the requests response and decode the content
-        response = requests.get(furl)
-        jresponse = response.json()
-        content = ""
         try:
-            binary_content = re.sub('\n','',jresponse['content'])
-            content = base64.b64decode(binary_content).decode('utf-8')
+            fpath = d['path']
+            furl = d['url']
+            fsha = d['sha']
+            _, fname = os.path.split(fpath)
+            _, fext = os.path.splitext(fpath)
+        except:
+            print(" > XXXXXXXX Failed to find file info.")
+            return
 
-        except KeyError:
-            print(" > XXXXXXXX Failed to extract 'content' field. You probably hit the rate limit.")
-            return 
-
-        # Now create the actual search index record
         indexed_time = clean_timestamp(datetime.now())
 
-        usable_url = "https://github.com/%s/blob/master/%s"%(repo_name, fpath)
+        if fext in MARKDOWN_EXTS:
+            print("Indexing markdown doc %s from repo %s"%(fname,repo_name))
 
-        # Add one document per issue thread,
-        # containing entire text of thread.
+            # Unpack the requests response and decode the content
+            # 
+            # don't forget the headers for private repos!
+            # useful: https://bit.ly/2LSAflS
+
+            headers = {'Authorization' : 'token %s'%(gh_token)}
+
+            response = requests.get(furl, headers=headers)
+            if response.status_code==200:
+                jresponse = response.json()
+                content = ""
+                try:
+                    binary_content = re.sub('\n','',jresponse['content'])
+                    content = base64.b64decode(binary_content).decode('utf-8')
+                except KeyError:
+                    print(" > XXXXXXXX Failed to extract 'content' field. You probably hit the rate limit.")
+
+            else:
+                print(" > XXXXXXXX Failed to reach file URL. There may be a problem with authentication/headers.")
+                return 
+
+            usable_url = "https://github.com/%s/blob/master/%s"%(repo_name, fpath)
+
+            # Now create the actual search index record
+            writer.add_document(
+                    id = fsha,
+                    kind = 'markdown',
+                    created_time = '',
+                    modified_time = '',
+                    indexed_time = indexed_time,
+                    title = fname,
+                    url = usable_url,
+                    mimetype='',
+                    owner_email='',
+                    owner_name='',
+                    repo_name = repo_name,
+                    repo_url = repo_url,
+                    github_user = '',
+                    issue_title = '',
+                    issue_url = '',
+                    content = content
+            )
+
+        else:
+            print("Indexing github file %s from repo %s"%(fname,repo_name))
+
+            key = fname+"_"+fsha
+
+            # Now create the actual search index record
+            writer.add_document(
+                    id = key,
+                    kind = 'ghfile',
+                    created_time = '',
+                    modified_time = '',
+                    indexed_time = indexed_time,
+                    title = fname,
+                    url = repo_url,
+                    mimetype='',
+                    owner_email='',
+                    owner_name='',
+                    repo_name = repo_name,
+                    repo_url = repo_url,
+                    github_user = '',
+                    issue_title = '',
+                    issue_url = '',
+                    content = ''
+            )
+
+
+
+
+    # ------------------------------
+    # Add a single github file 
+    # to a search index.
+
+    def add_emailthread(self, writer, d, config, update=True):
+        """
+        Use a Github file API record to add a filename
+        to the search index.
+        """
+        indexed_time = clean_timestamp(datetime.now())
+
+        # Now create the actual search index record
         writer.add_document(
-                id = fsha,
-                kind = 'markdown',
+                id = d['permalink'],
+                kind = 'emailthread',
                 created_time = '',
                 modified_time = '',
                 indexed_time = indexed_time,
-                title = fname,
-                url = usable_url,
+                title = d['subject'],
+                url = d['permalink'],
                 mimetype='',
                 owner_email='',
-                owner_name='',
-                repo_name = repo_name,
-                repo_url = repo_url,
+                owner_name=d['original_sender'],
+                repo_name = '',
+                repo_url = '',
                 github_user = '',
                 issue_title = '',
                 issue_url = '',
-                content = content
+                content = d['content']
         )
 
 
 
+
     # ------------------------------
     # Define how to update search index
     # using different kinds of collections
 
+
+    # ------------------------------
+    # Google Drive Files/Documents
+
     def update_index_gdocs(self, 
                            config):
         """
@@ -478,7 +612,7 @@ class Search:
         remote_ids = set()
         full_items = {}
         while True:
-            ps = 12
+            ps = 100
             results = drive.list(
                     pageSize=ps,
                     pageToken=nextPageToken,
@@ -496,11 +630,11 @@ class Search:
                 # Also store the doc
                 full_items[f['id']] = f
             
-            # Shorter:
-            break
-            ## Longer:
-            #if nextPageToken is None:
-            #    break
+            ## Shorter:
+            #break
+            # Longer:
+            if nextPageToken is None:
+                break
 
 
         writer = self.ix.writer()
@@ -544,13 +678,13 @@ class Search:
         print("Done, updated %d documents in the index" % count)
 
 
+    # ------------------------------
+    # Github Issues/Comments
 
-    def update_index_issues(self, gh_oauth_token, config):
+    def update_index_issues(self, gh_token, config):
         """
         Update the search index using a collection of 
         Github repo issues and comments.
-
-        gh_oauth_token can also be an access token.
         """
         # Updated algorithm:
         # - get set of indexed ids
@@ -562,7 +696,7 @@ class Search:
         # ------
         indexed_issues = set()
         p = QueryParser("kind", schema=self.ix.schema)
-        q = p.parse("gdoc")
+        q = p.parse("issue")
         with self.ix.searcher() as s:
             results = s.search(q,limit=None)
             for result in results:
@@ -572,25 +706,29 @@ class Search:
         # Get the set of remote ids:
         # ------
         # Start with api object
-        g = Github(gh_oauth_token)
+        g = Github(gh_token)
 
         # Now index all issue threads in the user-specified repos
 
+        # Start by collecting all the things
+        remote_issues = set()
+        full_items = {}
+
         # Iterate over each repo 
         list_of_repos = config['repositories']
         for r in list_of_repos:
 
-            # Start by collecting all the things
-            remote_issues = set()
-            full_items = {}
-
             if '/' not in r:
                 err = "Error: specify org/reponame or user/reponame in list of repos"
                 raise Exception(err)
 
             this_org, this_repo = re.split('/',r)
-            org = g.get_organization(this_org)
-            repo = org.get_repo(this_repo)
+            try:
+                org = g.get_organization(this_org)
+                repo = org.get_repo(this_repo)
+            except:
+                print("Error: could not gain access to repository %s"%(r))
+                continue
 
             # Iterate over each issue thread
             issues = repo.get_issues()
@@ -608,30 +746,15 @@ class Search:
         writer = self.ix.writer()
         count = 0
 
-        # Drop any issues in indexed_issues
-        # not in remote_issues
-        drop_issues = indexed_issues - remote_issues
-        for drop_issue in drop_issues:
+        # Drop issues in indexed_issues
+        for drop_issue in indexed_issues:
             writer.delete_by_term('id',drop_issue)
 
 
-        # Update any issue in indexed_issues
-        # and in remote_issues
-        update_issues = indexed_issues & remote_issues
-        for update_issue in update_issues:
-            # cop out
-            writer.delete_by_term('id',update_issue)
-            item = full_items[update_issue]
-            self.add_issue(writer, item, config, update=True)
-            count += 1
-
-
-        # Add any issue not in indexed_issues
-        # and in remote_issues
-        add_issues = remote_issues - indexed_issues
-        for add_issue in add_issues:
+        # Add any issue in remote_issues
+        for add_issue in remote_issues:
             item = full_items[add_issue]
-            self.add_issue(writer, item, config, update=False)
+            self.add_issue(writer, item, gh_token, config, update=False)
             count += 1
 
 
@@ -640,16 +763,15 @@ class Search:
 
 
 
+    # ------------------------------
+    # Github Files
 
-    def update_index_markdown(self, gh_oauth_token, config): 
+    def update_index_ghfiles(self, gh_token, config): 
         """
         Update the search index using a collection of 
-        Markdown files from a Github repo.
-
-        gh_oauth_token can also be an access token.
+        files (and, separately, Markdown files) from 
+        a Github repo.
         """
-        EXT = '.md'
-
         # Updated algorithm:
         # - get set of indexed ids
         # - get set of remote ids
@@ -660,6 +782,12 @@ class Search:
         # ------
         indexed_ids = set()
         p = QueryParser("kind", schema=self.ix.schema)
+        q = p.parse("ghfile")
+        with self.ix.searcher() as s:
+            results = s.search(q,limit=None)
+            for result in results:
+                indexed_ids.add(result['id'])
+
         q = p.parse("markdown")
         with self.ix.searcher() as s:
             results = s.search(q,limit=None)
@@ -669,94 +797,167 @@ class Search:
         # Get the set of remote ids:
         # ------
         # Start with api object
-        g = Github(gh_oauth_token)
+        g = Github(gh_token)
 
-        # Now index all markdown files
-        # in the user-specified repos
+        # Now index all the files.
+
+        # Start by collecting all the things
+        remote_ids = set()
+        full_items = {}
 
         # Iterate over each repo 
         list_of_repos = config['repositories']
         for r in list_of_repos:
 
-            # Start by collecting all the things
-            remote_ids = set()
-            full_items = {}
-
             if '/' not in r:
                 err = "Error: specify org/reponame or user/reponame in list of repos"
                 raise Exception(err)
 
             this_org, this_repo = re.split('/',r)
-            org = g.get_organization(this_org)
-            repo = org.get_repo(this_repo)
+            try:
+                org = g.get_organization(this_org)
+                repo = org.get_repo(this_repo)
+            except:
+                print("Error: could not gain access to repository %s"%(r))
+                continue
 
-            # ---------
-            # begin markdown-specific code
 
             # Get head commit
             commits = repo.get_commits()
-            last = commits[0]
-            sha = last.sha
+            try:
+                last = commits[0]
+                sha = last.sha
+            except GithubException:
+                print("Error: could not get commits from repository %s"%(r))
+                continue
 
             # Get all the docs
             tree = repo.get_git_tree(sha=sha, recursive=True)
             docs = tree.raw_data['tree']
+            print("Parsing file ids from repository %s"%(r))
 
             for d in docs:
 
                 # For each doc, get the file extension
-                # If it matches EXT, download the file
+                # and decide what to do with it.
+
                 fpath = d['path']
                 _, fname = os.path.split(fpath)
                 _, fext = os.path.splitext(fpath)
+                fpathpieces = fpath.split('/')
 
-                if fext==EXT:
+                ignore_file = fname[0]=='.' or fname[0]=='_'
+                ignore_dir = False
+                for piece in fpathpieces:
+                    if piece[0]=='.' or piece[0]=='_':
+                        ignore_dir = True
 
+                if not ignore_file and not ignore_dir:
                     key = d['sha']
+
                     d['org'] = this_org
                     d['repo'] = this_repo
                     value = d
 
-                    # Stash the doc for later
                     remote_ids.add(key)
                     full_items[key] = value
 
         writer = self.ix.writer()
         count = 0
 
-
         # Drop any id in indexed_ids
-        # not in remote_ids
-        drop_ids = indexed_ids - remote_ids
-        for drop_id in drop_ids:
+        for drop_id in indexed_ids:
             writer.delete_by_term('id',drop_id)
 
 
-        # Update any id in indexed_ids
+        # Add any issue in remote_ids
         # and in remote_ids
-        update_ids = indexed_ids & remote_ids
-        for update_id in update_ids:
-            # cop out
-            writer.delete_by_term('id',update_id)
-            item = full_items[update_id]
-            self.add_markdown(writer, item, config, update=True)
-            count += 1
-
-
-        # Add any issue not in indexed_ids
-        # and in remote_ids
-        add_ids = remote_ids - indexed_ids
-        for add_id in add_ids:
+        for add_id in remote_ids:
             item = full_items[add_id]
-            self.add_markdown(writer, item, config, update=False)
+            self.add_ghfile(writer, item, gh_token, config, update=False)
             count += 1
 
 
         writer.commit()
-        print("Done, updated %d markdown documents in the index" % count)
+        print("Done, updated %d Github files in the index" % count)
 
 
 
+    # ------------------------------
+    # Groups.io Emails
+
+
+    def update_index_emailthreads(self, groupsio_token, config):
+        """
+        Update the search index using the email archives
+        of groups.io groups. This method looks deceptively
+        simple, all the logic is hidden in the spider
+        (groupsio_util.py).
+
+        RELEASE THE SPIDER!!!
+        """
+        # Algorithm:
+        # - get set of indexed ids
+        # - get set of remote ids
+        # - drop indexed ids not in remote ids
+        # - index all remote ids
+
+        # Get the set of indexed ids:
+        # ------
+        indexed_ids = set()
+        p = QueryParser("kind", schema=self.ix.schema)
+        q = p.parse("emailthread")
+        with self.ix.searcher() as s:
+            results = s.search(q,limit=None)
+            for result in results:
+                indexed_ids.add(result['id'])
+
+        # Get the set of remote ids:
+        # ------
+        spider = GroupsIOArchivesCrawler(groupsio_token,'dcppc')
+
+        # ask spider to crawl the archives
+        spider.crawl_group_archives()
+
+        # now spider.archives is a list of dictionaries
+        # that each represent a thread:
+        #   thread = {
+        #           'permalink' : permalink,
+        #           'subject' : subject,
+        #           'original_sender' : original_sender,
+        #           'content' : full_content
+        #   }
+        #
+        # It is hard to reliablly extract more information
+        # than that from the email thread.
+
+        writer = self.ix.writer()
+        count = 0
+
+        # archives is a dictionary
+        # keys are IDs (urls)
+        # values are dictionaries
+        archives = spider.get_archives()
+
+        # Start by collecting all the things
+        remote_ids = set()
+        for k in archives.keys():
+            remote_ids.add(k)
+
+        # drop indexed_ids
+        for drop_id in indexed_ids:
+            writer.delete_by_term('id',drop_id)
+
+        # add remote_ids
+        for add_id in remote_ids:
+            item = archives[add_id]
+            self.add_emailthread(writer, item, config, update=False)
+            count += 1
+
+        writer.commit()
+        print("Done, updated %d Groups.io email threads in the index" % count)
+
+
     # ---------------------------------
     # Search results bundler
 
@@ -831,12 +1032,82 @@ class Search:
 
 
 
+    def cap(self, s, l):
+        return s if len(s) <= l else s[0:l - 3] + '...'
+
+    def get_document_total_count(self):
+        p = QueryParser("kind", schema=self.ix.schema)
+
+        counts = {
+                "gdoc" : None,
+                "issue" : None,
+                "ghfile" : None,
+                "markdown" : None,
+                "emailthread" : None,
+                "total" : None
+        }
+        for key in counts.keys():
+            q = p.parse(key)
+            with self.ix.searcher() as s:
+                results = s.search(q,limit=None)
+                counts[key] = len(results)
+
+        counts['total'] = sum(counts[k] for k in counts.keys())
+
+        return counts
+
+
+    def get_list(self,doctype):
+        """
+        Get a listing of all files, 
+        so we can construct the page that 
+        lists everyone and everything that
+        centillion indexes.
+        """
+        # Unfortunately, we have to treat
+        # each doctype separately, b/c of
+        # what is most relevant to display
+        # in the everything-list.
+        item_keys=''
+        if doctype=='gdoc':
+            item_keys = ['title','owner_name','url','mimetype','created_time','modified_time']
+        elif doctype=='issue':
+            item_keys = ['title','repo_name','repo_url','url','created_time','modified_time']
+        elif doctype=='emailthread':
+            item_keys = ['title','owner_name','url']
+        elif doctype=='ghfile':
+            item_keys = ['title','repo_name','repo_url','url']
+        elif doctype=='markdown':
+            item_keys = ['title','repo_name','repo_url','url']
+        else:
+            raise Exception("Could not find document of type %s"%(doctype))
+
+        json_results = []
+
+        p = QueryParser("kind", schema=self.ix.schema)
+        q = p.parse(doctype)
+        with self.ix.searcher() as s:
+            results = s.search(q,limit=None)
+            for r in results:
+                d = {}
+                for k in item_keys:
+                    if k=='created_time' or k=='modified_time':
+                        #d[k] = r[k]
+                        d[k] = dateutil.parser.parse(r[k]).strftime("%Y-%m-%d")
+                    else:
+                        d[k] = r[k]
+                json_results.append(d)
+
+        return json_results
+
+
+
     def search(self, query_list, fields=None):
 
         with self.ix.searcher() as searcher:
             query_string = " ".join(query_list)
             query = None
-            if "\"" in query_string or ":" in query_string:
+            if ":" in query_string:
                 query = QueryParser("content", self.schema).parse(query_string)
             elif len(fields) == 1 and fields[0] == "filename":
                 pass
@@ -845,8 +1116,7 @@ class Search:
             else:
                 # If the user does not specify a field,
                 # these are the fields that are actually searched
-                fields = ['title',
-                          'content']
+                fields = ['title', 'content','owner_name','owner_email','url']
             if not query:
                 query = MultifieldParser(fields, schema=self.ix.schema).parse(query_string)
             parsed_query = "%s" % query
@@ -858,37 +1128,11 @@ class Search:
 
 
 
-    def cap(self, s, l):
-        return s if len(s) <= l else s[0:l - 3] + '...'
-
-    def get_document_total_count(self):
-        p = QueryParser("kind", schema=self.ix.schema)
-
-        kind_labels = {
-                "documents" : "gdoc",
-                "markdown" :  "markdown",
-                "issues" :    "issue",
-        }
-        counts = {
-                "documents" : None,
-                "markdown" : None,
-                "issues" : None,
-                "total" : None
-        }
-        for key in kind_labels:
-            kind = kind_labels[key]
-            q = p.parse(kind)
-            with self.ix.searcher() as s:
-                results = s.search(q,limit=None)
-                counts[key] = len(results)
-
-        ## These two should NOT be different, but they are...
-        #counts['total'] = self.ix.searcher().doc_count_all()
-        counts['total'] = counts['documents'] + counts['markdown'] + counts['issues']
-
-        return counts
 
 if __name__ == "__main__":
+
+    raise Exception("Error: main method not implemented (fix groupsio credentials first)")
+
     search = Search("search_index")
 
     from get_centillion_config import get_centillion_config

config_centillion.example.py

@@ -0,0 +1,28 @@
+config = {
+    "repositories" : [
+        "dcppc/project-management",
+        "dcppc/nih-demo-meetings",
+        "dcppc/internal",
+        "dcppc/organize",
+        "dcppc/dcppc-bot",
+        "dcppc/full-stacks",
+        "dcppc/design-guidelines-discuss",
+        "dcppc/dcppc-deliverables",
+        "dcppc/dcppc-milestones",
+        "dcppc/crosscut-metadata",
+        "dcppc/lucky-penny",
+        "dcppc/dcppc-workshops",
+        "dcppc/metadata-matrix",
+        "dcppc/data-stewards",
+        "dcppc/dcppc-phase1-demos",
+        "dcppc/apis",
+        "dcppc/2018-june-workshop",
+        "dcppc/2018-july-workshop",
+        "dcppc/2018-august-workshop",
+        "dcppc/2018-september-workshop",
+        "dcppc/design-guidelines",
+        "dcppc/2018-may-workshop",
+        "dcppc/centillion"
+    ]
+}
+

config_centillion.json

@@ -1,6 +0,0 @@
-{
-    "repositories" : [
-        "dcppc/2018-june-workshop",
-        "dcppc/2018-july-workshop"
-    ]
-}

config_flask.example.py

@@ -2,8 +2,9 @@
 INDEX_DIR = "search_index"
 
 # oauth client deets
-GITHUB_OAUTH_CLIENT_ID = "63f8d49c651840cbe31e"
-GITHUB_OAUTH_CLIENT_SECRET = "36d9a4611f7427336d3c89ed041c45d086b793ee"
+GITHUB_OAUTH_CLIENT_ID = "XXX"
+GITHUB_OAUTH_CLIENT_SECRET = "YYY"
+GITHUB_TOKEN = "ZZZ"
 
 # More information footer: Repository label
 FOOTER_REPO_ORG = "charlesreid1"
@@ -12,8 +13,8 @@ FOOTER_REPO_NAME = "centillion"
 # Toggle to show Whoosh parsed query
 SHOW_PARSED_QUERY=True
 
-TAGLINE = "Search all the things"
+TAGLINE = "Search All The Things"
 
 # Flask settings
 DEBUG = True
-SECRET_KEY = '42c5a8eda356ca9d9c3ab2d149541e6b91d843fa'
+SECRET_KEY = 'WWWWW'

gdrive_util.py

@@ -29,8 +29,7 @@ class GDrive(object):
     ):
         """
         Set up the Google Drive API instance.
-        Factory method: create it and hand it over.
-        Then we're finished.
+        Factory method: create it here, hand it over in get_service().
         """
         self.credentials_file = credentials_file
         self.client_secret_file = client_secret_file
@@ -40,6 +39,9 @@ class GDrive(object):
         self.store = file.Storage(credentials_file)
 
     def get_service(self):
+        """
+        Return an instance of the Google Drive API service.
+        """
 
         creds = self.store.get()
         if not creds or creds.invalid:
@@ -49,3 +51,6 @@ class GDrive(object):
         service = build('drive', 'v3', http=creds.authorize(Http()))
         return service
 
+if __name__=="__main__":
+    g = GDrive()
+    s = g.get_service()

get_centillion_config.py

@@ -1,12 +0,0 @@
-import json
-
-CONFIG_FILE = 'config_centillion.json'
-
-def get_centillion_config(filename=CONFIG_FILE):
-    """
-    Load the centillion configuration
-    """
-    with open(filename,'r') as f:
-        d = json.load(f)
-    return d
-

groupsio_util.py

@@ -0,0 +1,383 @@
+import requests, os, re
+from bs4 import BeautifulSoup
+
+class GroupsIOException(Exception):
+    pass
+
+class GroupsIOArchivesCrawler(object):
+    """
+    This is a Groups.io spider
+    designed to crawl the email
+    archives of a group.
+
+    credentials (dictionary):
+        groupsio_token :     api access token
+        groupsio_username     :     username
+        groupsio_password     :     password
+    """
+    def __init__(self,
+                 credentials,
+                 group_name):
+        # template url for archives page (list of topics)
+        self.url = "https://{group}.groups.io/g/{subgroup}/topics"
+        self.login_url = "https://groups.io/login"
+
+        self.credentials = credentials
+        self.group_name = group_name
+        self.crawled_archives = False
+        self.archives = None
+
+
+    def get_archives(self):
+        """
+        Return a list of dictionaries containing 
+        information about each email topic in the 
+        groups.io email archive.
+
+        Call crawl_group_archives() first!
+        """
+        return self.archives
+
+
+    def get_subgroups_list(self):
+        """
+        Use the API to get a list of subgroups.
+        """
+        subgroups_url = 'https://api.groups.io/v1/getsubgroups'
+
+        key = self.credentials['groupsio_token']
+
+        data = [('group_name', self.group_name),
+                ('limit',100)
+        ]
+        response = requests.post(subgroups_url,
+                                 data=data,
+                                 auth=(key,''))
+        response = response.json()
+        data = response['data']
+
+        subgroups = {}
+        for group in data:
+            k = group['id']
+            v = re.sub(r'dcppc\+','',group['name'])
+            subgroups[k] = v
+
+            ## Short circuit
+            ## for debugging purposes
+            #break
+
+        return subgroups
+
+
+    def crawl_group_archives(self):
+        """
+        Spider will crawl the email archives of the entire group
+        by crawling the email archives of each subgroup.
+        """
+        self.archives = {}
+
+        subgroups = self.get_subgroups_list()
+
+        # ------------------------------
+        # Start by logging in.
+
+        # Create session object to persist session data
+        session = requests.Session()
+
+        # Log in to the website
+        data = dict(email = self.credentials['groupsio_username'],
+                    password = self.credentials['groupsio_password'],
+                    timezone = 'America/Los_Angeles')
+
+        r = session.post(self.login_url,
+                         data = data)
+
+        csrf = self.get_csrf(r)
+
+        # ------------------------------
+        # For each subgroup, crawl the archives
+        # and return a list of dictionaries
+        # containing all the email threads.
+        for subgroup_id in subgroups.keys():
+            self.crawl_subgroup_archives(session,
+                                         csrf,
+                                         subgroup_id, 
+                                         subgroups[subgroup_id])
+
+        # Done. archives are now tucked away
+        # in the variable self.archives
+        # 
+        # self.archives is a dictionary of dictionaries,
+        # with each key a URL and each value a dictionary
+        # containing info about a thread.
+        # ------------------------------
+
+
+
+
+    def crawl_subgroup_archives(self, session, csrf, subgroup_id, subgroup_name):
+        """
+        This kicks off the process to crawl the entire
+        archives of a given subgroup on groups.io.
+
+        For a given subgroup the url is self.url,
+        
+            https://{group}.groups.io/g/{subgroup}/topics
+
+        This is the first of a paginated list of topics.
+        Procedure is:
+        - passed a starting page (or its contents)
+        - iterate through all topics via the HTML page elements
+        - assemble a bundle of information about each topic:
+            - topic title, by, URL, date, content, permalink
+            - content filtering:
+                - ^From, Reply-To, Date, To, Subject
+                - Lines containing phone numbers
+                    - 9 digits
+                    - XXX-XXX-XXXX, (XXX) XXX-XXXX
+                    - XXXXXXXXXX, XXX XXX XXXX
+                    - ^Work: or (Work) or Work$
+                    - Home, Cell, Mobile
+                    - +1 XXX 
+                    - \w@\w
+        - while next button is not greyed out,
+        - click the next button
+
+        everything stored in self.archives:
+        list of dictionaries.
+
+        """
+        prefix = "https://{group}.groups.io".format(group=self.group_name)
+
+        url = self.url.format(group=self.group_name, 
+                              subgroup=subgroup_name)
+
+        # ------------------------------
+
+        # Now get the first page
+        r = session.get(url)
+
+        # ------------------------------
+        # Fencepost algorithm:
+
+        # First page:
+
+        # Extract a list of (title, link) items
+        items = self.extract_archive_page_items_(r)
+
+        # Get the next link
+        next_url = self.get_next_url_(r)
+
+        # Now add each item to the archive of threads,
+        # then find the next button.
+        self.add_items_to_archives_(session,subgroup_name,items)
+
+        if next_url is None:
+            return
+        else:
+            full_next_url = prefix + next_url
+
+        # Now click the next button
+        next_request = requests.get(full_next_url)
+
+        while next_request.status_code==200:
+            items = self.extract_archive_page_items_(next_request)
+            next_url = self.get_next_url_(next_request)
+            self.add_items_to_archives_(session,subgroup_name,items)
+            if next_url is None:
+                return
+            else:
+                full_next_url = prefix + next_url
+            next_request = requests.get(full_next_url)
+        
+
+
+    def add_items_to_archives_(self,session,subgroup_name,items):
+        """
+        Given a set of items from a list of threads,
+        items being title and link,
+        get the page and store all info
+        in self.archives variable
+        (list of dictionaries)
+        """
+        for (title, link) in items:
+            # Get the thread page:
+            prefix = "https://{group}.groups.io".format(group=self.group_name)
+            full_link = prefix + link
+            r = session.get(full_link)
+            soup = BeautifulSoup(r.text,'html.parser')
+
+            # soup contains the entire thread
+
+            # What are we extracting:
+            # 1. thread number
+            # 2. permalink
+            # 3. content/text (filtered)
+
+            # - - - - - - - - - - - - - - 
+            # 1. topic/thread number:
+            # <a rel="nofollow" href="">
+            # where link is:
+            # https://{group}.groups.io/g/{subgroup}/topic/{topic_id}
+            # example topic id: 24209140
+            #
+            # ugly links are in the form 
+            # https://dcppc.groups.io/g/{subgroup}/topic/some_text_here/{thread_id}?p=,,,,,1,2,3,,,4,,5
+            # split at ?, 0th portion
+            # then split at /, last (-1th) portion
+            topic_id = link.split('?')[0].split('/')[-1]
+
+            # - - - - - - - - - - - - - - - 
+            # 2. permalink:
+            # - current link is ugly link
+            # - permalink is the nice one
+            # - topic id is available from the ugly link
+            # https://{group}.groups.io/g/{subgroup}/topic/{topic_id}
+
+            permalink_template = "https://{group}.groups.io/g/{subgroup}/topic/{topic_id}"
+            permalink = permalink_template.format(
+                    group = self.group_name,
+                    subgroup = subgroup_name, 
+                    topic_id = topic_id
+            )
+
+            # - - - - - - - - - - - - - - - 
+            # 3. content:
+
+            # Need to rearrange how we're assembling threads here.
+            # This is one thread, no?
+            content = []
+
+            subject = soup.find('title').text
+
+            # Extract information for the schema:
+            # - permalink for thread (done)
+            # - subject/title (done)
+            # - original sender email/name (done)
+            # - content (done)
+
+            # Groups.io pages have zero CSS classes, which makes everything
+            # a giant pain in the neck to interact with. Thanks Groups.io!
+            original_sender = ''
+            for i, tr in enumerate(soup.find_all('tr',{'class':'test'})):
+                # Every other tr row contains an email.
+                if (i+1)%2==0:
+                    # nope, no email here
+                    pass
+                else:
+                    # found an email!
+                    # this is a maze, thanks groups.io
+                    td = tr.find('td')
+                    divrow = td.find('div',{'class':'row'}).find('div',{'class':'pull-left'})
+                    if (i+1)==1:
+                        original_sender = divrow.text.strip()
+                    for div in td.find_all('div'):
+                        if div.has_attr('id'):
+
+                            # purge any signatures
+                            for x in div.find_all('div',{'id':'Signature'}):
+                                x.extract()
+
+                            # purge any headers
+                            for x in div.find_all('div'): 
+                                nonos = ['From:','Sent:','To:','Cc:','CC:','Subject:']
+                                for nono in nonos:
+                                    if nono in x.text:
+                                        x.extract()
+
+                            message_text = div.get_text()
+
+                            # More filtering:
+
+                            # phone numbers
+                            message_text = re.sub(r'[0-9]{3}-[0-9]{3}-[0-9]{4}','XXX-XXX-XXXX',message_text)
+                            message_text = re.sub(r'[0-9]\{10\}','XXXXXXXXXX',message_text)
+
+                            content.append(message_text)
+
+            full_content = "\n".join(content)
+
+            thread = {
+                    'permalink' : permalink,
+                    'subject' : subject,
+                    'original_sender' : original_sender,
+                    'content' : full_content
+            }
+
+            print(" + Archiving thread: %s"%(thread['subject']))
+            self.archives[permalink] = thread
+
+
+    def extract_archive_page_items_(self, response):
+        """
+        (Private method)
+
+        Given a response from a GET request,
+        use beautifulsoup to extract all items
+        (thread titles and ugly thread links)
+        and pass them back in a list.
+        """
+        soup = BeautifulSoup(response.content,"html.parser")
+        rows = soup.find_all('tr',{'class':'test'})
+        if 'rate limited' in soup.text:
+            raise GroupsIOException("Error: rate limit in place for Groups.io")
+
+        results = []
+        for row in rows:
+            # We don't care about anything except title and ugly link
+            subject = row.find('span',{'class':'subject'})
+            title = subject.get_text()
+            link = row.find('a')['href']
+            #print(title)
+            results.append((title,link))
+
+        return results
+
+
+    def get_next_url_(self, response):
+        """
+        (Private method)
+
+        Given a response (which is a list of threads),
+        find the next button and return the URL.
+
+        If no next URL, if is disabled, then return None.
+        """
+        soup = BeautifulSoup(response.text,'html.parser')
+        chevron = soup.find('i',{'class':'fa-chevron-right'})
+        try:
+            if '#' in chevron.parent['href']:
+                # empty link, abort
+                return None
+        except AttributeError:
+            # I don't even now
+            return None
+
+        if chevron.parent.parent.has_attr('class') and 'disabled' in chevron.parent.parent['class']:
+            # no next link, abort
+            return None
+
+        return chevron.parent['href']
+
+
+
+    def get_csrf(self,resp):
+        """
+        Find the CSRF token embedded in the subgroup page
+        """
+        soup = BeautifulSoup(resp.text,'html.parser')
+        csrf = ''
+        for i in soup.find_all('input'):
+            # Note that i.name is different from i['name']
+            # the first is the actual tag,
+            # the second is the attribute name="xyz"
+            if i['name']=='csrf':
+                csrf = i['value']
+        
+        if csrf=='':
+            err = "ERROR: Could not find csrf token on page."
+            raise GroupsIOException(err)
+    
+        return csrf
+
+

requirements.txt

@@ -10,3 +10,5 @@ pypandoc>=1.4
 requests>=2.19
 pandoc>=1.0
 flask-dance>=1.0.0
+beautifulsoup4>=4.6
+python-dateutil>=2.6

static/dataTables.bootstrap.min.css

@@ -0,0 +1 @@
+table.dataTable{clear:both;margin-top:6px !important;margin-bottom:6px !important;max-width:none !important;border-collapse:separate !important}table.dataTable td,table.dataTable th{-webkit-box-sizing:content-box;box-sizing:content-box}table.dataTable td.dataTables_empty,table.dataTable th.dataTables_empty{text-align:center}table.dataTable.nowrap th,table.dataTable.nowrap td{white-space:nowrap}div.dataTables_wrapper div.dataTables_length label{font-weight:normal;text-align:left;white-space:nowrap}div.dataTables_wrapper div.dataTables_length select{width:75px;display:inline-block}div.dataTables_wrapper div.dataTables_filter{text-align:right}div.dataTables_wrapper div.dataTables_filter label{font-weight:normal;white-space:nowrap;text-align:left}div.dataTables_wrapper div.dataTables_filter input{margin-left:0.5em;display:inline-block;width:auto}div.dataTables_wrapper div.dataTables_info{padding-top:8px;white-space:nowrap}div.dataTables_wrapper div.dataTables_paginate{margin:0;white-space:nowrap;text-align:right}div.dataTables_wrapper div.dataTables_paginate ul.pagination{margin:2px 0;white-space:nowrap}div.dataTables_wrapper div.dataTables_processing{position:absolute;top:50%;left:50%;width:200px;margin-left:-100px;margin-top:-26px;text-align:center;padding:1em 0}table.dataTable thead>tr>th.sorting_asc,table.dataTable thead>tr>th.sorting_desc,table.dataTable thead>tr>th.sorting,table.dataTable thead>tr>td.sorting_asc,table.dataTable thead>tr>td.sorting_desc,table.dataTable thead>tr>td.sorting{padding-right:30px}table.dataTable thead>tr>th:active,table.dataTable thead>tr>td:active{outline:none}table.dataTable thead .sorting,table.dataTable thead .sorting_asc,table.dataTable thead .sorting_desc,table.dataTable thead .sorting_asc_disabled,table.dataTable thead .sorting_desc_disabled{cursor:pointer;position:relative}table.dataTable thead .sorting:after,table.dataTable thead .sorting_asc:after,table.dataTable thead .sorting_desc:after,table.dataTable thead .sorting_asc_disabled:after,table.dataTable thead .sorting_desc_disabled:after{position:absolute;bottom:8px;right:8px;display:block;font-family:'Glyphicons Halflings';opacity:0.5}table.dataTable thead .sorting:after{opacity:0.2;content:"---"}table.dataTable thead .sorting_asc:after{content:"/\\"}table.dataTable thead .sorting_desc:after{content:"\\/"}table.dataTable thead .sorting_asc_disabled:after,table.dataTable thead .sorting_desc_disabled:after{color:#eee}div.dataTables_scrollHead table.dataTable{margin-bottom:0 !important}div.dataTables_scrollBody>table{border-top:none;margin-top:0 !important;margin-bottom:0 !important}div.dataTables_scrollBody>table>thead .sorting:after,div.dataTables_scrollBody>table>thead .sorting_asc:after,div.dataTables_scrollBody>table>thead .sorting_desc:after{display:none}div.dataTables_scrollBody>table>tbody>tr:first-child>th,div.dataTables_scrollBody>table>tbody>tr:first-child>td{border-top:none}div.dataTables_scrollFoot>.dataTables_scrollFootInner{box-sizing:content-box}div.dataTables_scrollFoot>.dataTables_scrollFootInner>table{margin-top:0 !important;border-top:none}@media screen and (max-width: 767px){div.dataTables_wrapper div.dataTables_length,div.dataTables_wrapper div.dataTables_filter,div.dataTables_wrapper div.dataTables_info,div.dataTables_wrapper div.dataTables_paginate{text-align:center}}table.dataTable.table-condensed>thead>tr>th{padding-right:20px}table.dataTable.table-condensed .sorting:after,table.dataTable.table-condensed .sorting_asc:after,table.dataTable.table-condensed .sorting_desc:after{top:6px;right:6px}table.table-bordered.dataTable th,table.table-bordered.dataTable td{border-left-width:0}table.table-bordered.dataTable th:last-child,table.table-bordered.dataTable th:last-child,table.table-bordered.dataTable td:last-child,table.table-bordered.dataTable td:last-child{border-right-width:0}table.table-bordered.dataTable tbody th,table.table-bordered.dataTable tbody td{border-bottom-width:0}div.dataTables_scrollHead table.table-bordered{border-bottom-width:0}div.table-responsive>div.dataTables_wrapper>div.row{margin:0}div.table-responsive>div.dataTables_wrapper>div.row>div[class^="col-"]:first-child{padding-left:0}div.table-responsive>div.dataTables_wrapper>div.row>div[class^="col-"]:last-child{padding-right:0}

static/dataTables.bootstrap.min.js

@@ -0,0 +1,8 @@
+/*!
+ DataTables Bootstrap 3 integration
+ ©2011-2014 SpryMedia Ltd - datatables.net/license
+*/
+(function(){var f=function(c,b){c.extend(!0,b.defaults,{dom:"<'row'<'col-sm-6'l><'col-sm-6'f>><'row'<'col-sm-12'tr>><'row'<'col-sm-6'i><'col-sm-6'p>>",renderer:"bootstrap"});c.extend(b.ext.classes,{sWrapper:"dataTables_wrapper form-inline dt-bootstrap",sFilterInput:"form-control input-sm",sLengthSelect:"form-control input-sm"});b.ext.renderer.pageButton.bootstrap=function(g,f,p,k,h,l){var q=new b.Api(g),r=g.oClasses,i=g.oLanguage.oPaginate,d,e,o=function(b,f){var j,m,n,a,k=function(a){a.preventDefault();
+c(a.currentTarget).hasClass("disabled")||q.page(a.data.action).draw(!1)};j=0;for(m=f.length;j<m;j++)if(a=f[j],c.isArray(a))o(b,a);else{e=d="";switch(a){case "ellipsis":d="&hellip;";e="disabled";break;case "first":d=i.sFirst;e=a+(0<h?"":" disabled");break;case "previous":d=i.sPrevious;e=a+(0<h?"":" disabled");break;case "next":d=i.sNext;e=a+(h<l-1?"":" disabled");break;case "last":d=i.sLast;e=a+(h<l-1?"":" disabled");break;default:d=a+1,e=h===a?"active":""}d&&(n=c("<li>",{"class":r.sPageButton+" "+
+e,"aria-controls":g.sTableId,tabindex:g.iTabIndex,id:0===p&&"string"===typeof a?g.sTableId+"_"+a:null}).append(c("<a>",{href:"#"}).html(d)).appendTo(b),g.oApi._fnBindAction(n,{action:a},k))}};o(c(f).empty().html('<ul class="pagination"/>').children("ul"),k)};b.TableTools&&(c.extend(!0,b.TableTools.classes,{container:"DTTT btn-group",buttons:{normal:"btn btn-default",disabled:"disabled"},collection:{container:"DTTT_dropdown dropdown-menu",buttons:{normal:"",disabled:"disabled"}},print:{info:"DTTT_print_info"},
+select:{row:"active"}}),c.extend(!0,b.TableTools.DEFAULTS.oTags,{collection:{container:"ul",button:"li",liner:"a"}}))};"function"===typeof define&&define.amd?define(["jquery","datatables"],f):"object"===typeof exports?f(require("jquery"),require("datatables")):jQuery&&f(jQuery,jQuery.fn.dataTable)})(window,document);

static/dataTables.responsive.css

@@ -0,0 +1,106 @@
+table.dataTable.dtr-inline.collapsed > tbody > tr > td:first-child,
+table.dataTable.dtr-inline.collapsed > tbody > tr > th:first-child {
+  position: relative;
+  padding-left: 30px;
+  cursor: pointer;
+}
+table.dataTable.dtr-inline.collapsed > tbody > tr > td:first-child:before,
+table.dataTable.dtr-inline.collapsed > tbody > tr > th:first-child:before {
+  top: 8px;
+  left: 4px;
+  height: 16px;
+  width: 16px;
+  display: block;
+  position: absolute;
+  color: white;
+  border: 2px solid white;
+  border-radius: 16px;
+  text-align: center;
+  line-height: 14px;
+  box-shadow: 0 0 3px #444;
+  box-sizing: content-box;
+  content: '+';
+  background-color: #31b131;
+}
+table.dataTable.dtr-inline.collapsed > tbody > tr > td:first-child.dataTables_empty:before,
+table.dataTable.dtr-inline.collapsed > tbody > tr > th:first-child.dataTables_empty:before {
+  display: none;
+}
+table.dataTable.dtr-inline.collapsed > tbody > tr.parent > td:first-child:before,
+table.dataTable.dtr-inline.collapsed > tbody > tr.parent > th:first-child:before {
+  content: '-';
+  background-color: #d33333;
+}
+table.dataTable.dtr-inline.collapsed > tbody > tr.child td:before {
+  display: none;
+}
+table.dataTable.dtr-inline.collapsed.compact > tbody > tr > td:first-child,
+table.dataTable.dtr-inline.collapsed.compact > tbody > tr > th:first-child {
+  padding-left: 27px;
+}
+table.dataTable.dtr-inline.collapsed.compact > tbody > tr > td:first-child:before,
+table.dataTable.dtr-inline.collapsed.compact > tbody > tr > th:first-child:before {
+  top: 5px;
+  left: 4px;
+  height: 14px;
+  width: 14px;
+  border-radius: 14px;
+  line-height: 12px;
+}
+table.dataTable.dtr-column > tbody > tr > td.control,
+table.dataTable.dtr-column > tbody > tr > th.control {
+  position: relative;
+  cursor: pointer;
+}
+table.dataTable.dtr-column > tbody > tr > td.control:before,
+table.dataTable.dtr-column > tbody > tr > th.control:before {
+  top: 50%;
+  left: 50%;
+  height: 16px;
+  width: 16px;
+  margin-top: -10px;
+  margin-left: -10px;
+  display: block;
+  position: absolute;
+  color: white;
+  border: 2px solid white;
+  border-radius: 16px;
+  text-align: center;
+  line-height: 14px;
+  box-shadow: 0 0 3px #444;
+  box-sizing: content-box;
+  content: '+';
+  background-color: #31b131;
+}
+table.dataTable.dtr-column > tbody > tr.parent td.control:before,
+table.dataTable.dtr-column > tbody > tr.parent th.control:before {
+  content: '-';
+  background-color: #d33333;
+}
+table.dataTable > tbody > tr.child {
+  padding: 0.5em 1em;
+}
+table.dataTable > tbody > tr.child:hover {
+  background: transparent !important;
+}
+table.dataTable > tbody > tr.child ul {
+  display: inline-block;
+  list-style-type: none;
+  margin: 0;
+  padding: 0;
+}
+table.dataTable > tbody > tr.child ul li {
+  border-bottom: 1px solid #efefef;
+  padding: 0.5em 0;
+}
+table.dataTable > tbody > tr.child ul li:first-child {
+  padding-top: 0;
+}
+table.dataTable > tbody > tr.child ul li:last-child {
+  border-bottom: none;
+}
+table.dataTable > tbody > tr.child span.dtr-title {
+  display: inline-block;
+  min-width: 75px;
+  font-weight: bold;
+}

static/dataTables.responsive.js

@@ -0,0 +1,873 @@
+/*! Responsive 1.0.6
+ * 2014-2015 SpryMedia Ltd - datatables.net/license
+ */
+
+/**
+ * @summary     Responsive
+ * @description Responsive tables plug-in for DataTables
+ * @version     1.0.6
+ * @file        dataTables.responsive.js
+ * @author      SpryMedia Ltd (www.sprymedia.co.uk)
+ * @contact     www.sprymedia.co.uk/contact
+ * @copyright   Copyright 2014-2015 SpryMedia Ltd.
+ *
+ * This source file is free software, available under the following license:
+ *   MIT license - http://datatables.net/license/mit
+ *
+ * This source file is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ * or FITNESS FOR A PARTICULAR PURPOSE. See the license files for details.
+ *
+ * For details please refer to: http://www.datatables.net
+ */
+
+(function(window, document, undefined) {
+
+
+var factory = function( $, DataTable ) {
+"use strict";
+
+/**
+ * Responsive is a plug-in for the DataTables library that makes use of
+ * DataTables' ability to change the visibility of columns, changing the
+ * visibility of columns so the displayed columns fit into the table container.
+ * The end result is that complex tables will be dynamically adjusted to fit
+ * into the viewport, be it on a desktop, tablet or mobile browser.
+ *
+ * Responsive for DataTables has two modes of operation, which can used
+ * individually or combined:
+ *
+ * * Class name based control - columns assigned class names that match the
+ *   breakpoint logic can be shown / hidden as required for each breakpoint.
+ * * Automatic control - columns are automatically hidden when there is no
+ *   room left to display them. Columns removed from the right.
+ *
+ * In additional to column visibility control, Responsive also has built into
+ * options to use DataTables' child row display to show / hide the information
+ * from the table that has been hidden. There are also two modes of operation
+ * for this child row display:
+ *
+ * * Inline - when the control element that the user can use to show / hide
+ *   child rows is displayed inside the first column of the table.
+ * * Column - where a whole column is dedicated to be the show / hide control.
+ *
+ * Initialisation of Responsive is performed by:
+ *
+ * * Adding the class `responsive` or `dt-responsive` to the table. In this case
+ *   Responsive will automatically be initialised with the default configuration
+ *   options when the DataTable is created.
+ * * Using the `responsive` option in the DataTables configuration options. This
+ *   can also be used to specify the configuration options, or simply set to
+ *   `true` to use the defaults.
+ *
+ *  @class
+ *  @param {object} settings DataTables settings object for the host table
+ *  @param {object} [opts] Configuration options
+ *  @requires jQuery 1.7+
+ *  @requires DataTables 1.10.1+
+ *
+ *  @example
+ *      $('#example').DataTable( {
+ *        responsive: true
+ *      } );
+ *    } );
+ */
+var Responsive = function ( settings, opts ) {
+	// Sanity check that we are using DataTables 1.10 or newer
+	if ( ! DataTable.versionCheck || ! DataTable.versionCheck( '1.10.1' ) ) {
+		throw 'DataTables Responsive requires DataTables 1.10.1 or newer';
+	}
+
+	this.s = {
+		dt: new DataTable.Api( settings ),
+		columns: []
+	};
+
+	// Check if responsive has already been initialised on this table
+	if ( this.s.dt.settings()[0].responsive ) {
+		return;
+	}
+
+	// details is an object, but for simplicity the user can give it as a string
+	if ( opts && typeof opts.details === 'string' ) {
+		opts.details = { type: opts.details };
+	}
+
+	this.c = $.extend( true, {}, Responsive.defaults, DataTable.defaults.responsive, opts );
+	settings.responsive = this;
+	this._constructor();
+};
+
+Responsive.prototype = {
+	/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+	 * Constructor
+	 */
+
+	/**
+	 * Initialise the Responsive instance
+	 *
+	 * @private
+	 */
+	_constructor: function ()
+	{
+		var that = this;
+		var dt = this.s.dt;
+
+		dt.settings()[0]._responsive = this;
+
+		// Use DataTables' private throttle function to avoid processor thrashing
+		$(window).on( 'resize.dtr orientationchange.dtr', dt.settings()[0].oApi._fnThrottle( function () {
+			that._resize();
+		} ) );
+
+		// Destroy event handler
+		dt.on( 'destroy.dtr', function () {
+			$(window).off( 'resize.dtr orientationchange.dtr draw.dtr' );
+		} );
+
+		// Reorder the breakpoints array here in case they have been added out
+		// of order
+		this.c.breakpoints.sort( function (a, b) {
+			return a.width < b.width ? 1 :
+				a.width > b.width ? -1 : 0;
+		} );
+
+		// Determine which columns are already hidden, and should therefore
+		// remain hidden. todo - should this be done? See thread 22677
+		//
+		// this.s.alwaysHidden = dt.columns(':hidden').indexes();
+
+		this._classLogic();
+		this._resizeAuto();
+
+		// Details handler
+		var details = this.c.details;
+		if ( details.type ) {
+			that._detailsInit();
+			this._detailsVis();
+
+			dt.on( 'column-visibility.dtr', function () {
+				that._detailsVis();
+			} );
+
+			// Redraw the details box on each draw. This is used until
+			// DataTables implements a native `updated` event for rows
+			dt.on( 'draw.dtr', function () {
+				dt.rows( {page: 'current'} ).iterator( 'row', function ( settings, idx ) {
+					var row = dt.row( idx );
+
+					if ( row.child.isShown() ) {
+						var info = that.c.details.renderer( dt, idx );
+						row.child( info, 'child' ).show();
+					}
+				} );
+			} );
+
+			$(dt.table().node()).addClass( 'dtr-'+details.type );
+		}
+
+		// First pass - draw the table for the current viewport size
+		this._resize();
+	},
+
+
+	/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+	 * Private methods
+	 */
+
+	/**
+	 * Calculate the visibility for the columns in a table for a given
+	 * breakpoint. The result is pre-determined based on the class logic if
+	 * class names are used to control all columns, but the width of the table
+	 * is also used if there are columns which are to be automatically shown
+	 * and hidden.
+	 *
+	 * @param  {string} breakpoint Breakpoint name to use for the calculation
+	 * @return {array} Array of boolean values initiating the visibility of each
+	 *   column.
+	 *  @private
+	 */
+	_columnsVisiblity: function ( breakpoint )
+	{
+		var dt = this.s.dt;
+		var columns = this.s.columns;
+		var i, ien;
+
+		// Class logic - determine which columns are in this breakpoint based
+		// on the classes. If no class control (i.e. `auto`) then `-` is used
+		// to indicate this to the rest of the function
+		var display = $.map( columns, function ( col ) {
+			return col.auto && col.minWidth === null ?
+				false :
+				col.auto === true ?
+					'-' :
+					$.inArray( breakpoint, col.includeIn ) !== -1;
+		} );
+
+		// Auto column control - first pass: how much width is taken by the
+		// ones that must be included from the non-auto columns
+		var requiredWidth = 0;
+		for ( i=0, ien=display.length ; i<ien ; i++ ) {
+			if ( display[i] === true ) {
+				requiredWidth += columns[i].minWidth;
+			}
+		}
+
+		// Second pass, use up any remaining width for other columns. For
+		// scrolling tables we need to subtract the width of the scrollbar. It
+		// may not be requires which makes this sub-optimal, but it would
+		// require another full redraw to make complete use of those extra few
+		// pixels
+		var scrolling = dt.settings()[0].oScroll;
+		var bar = scrolling.sY || scrolling.sX ? scrolling.iBarWidth : 0;
+		var widthAvailable = dt.table().container().offsetWidth - bar;
+		var usedWidth = widthAvailable - requiredWidth;
+
+		// Control column needs to always be included. This makes it sub-
+		// optimal in terms of using the available with, but to stop layout
+		// thrashing or overflow. Also we need to account for the control column
+		// width first so we know how much width is available for the other
+		// columns, since the control column might not be the first one shown
+		for ( i=0, ien=display.length ; i<ien ; i++ ) {
+			if ( columns[i].control ) {
+				usedWidth -= columns[i].minWidth;
+			}
+		}
+
+		// Allow columns to be shown (counting from the left) until we run out
+		// of room
+		var empty = false;
+		for ( i=0, ien=display.length ; i<ien ; i++ ) {
+			if ( display[i] === '-' && ! columns[i].control ) {
+				// Once we've found a column that won't fit we don't let any
+				// others display either, or columns might disappear in the
+				// middle of the table
+				if ( empty || usedWidth - columns[i].minWidth < 0 ) {
+					empty = true;
+					display[i] = false;
+				}
+				else {
+					display[i] = true;
+				}
+
+				usedWidth -= columns[i].minWidth;
+			}
+		}
+
+		// Determine if the 'control' column should be shown (if there is one).
+		// This is the case when there is a hidden column (that is not the
+		// control column). The two loops look inefficient here, but they are
+		// trivial and will fly through. We need to know the outcome from the
+		// first , before the action in the second can be taken
+		var showControl = false;
+
+		for ( i=0, ien=columns.length ; i<ien ; i++ ) {
+			if ( ! columns[i].control && ! columns[i].never && ! display[i] ) {
+				showControl = true;
+				break;
+			}
+		}
+
+		for ( i=0, ien=columns.length ; i<ien ; i++ ) {
+			if ( columns[i].control ) {
+				display[i] = showControl;
+			}
+		}
+
+		// Finally we need to make sure that there is at least one column that
+		// is visible
+		if ( $.inArray( true, display ) === -1 ) {
+			display[0] = true;
+		}
+
+		return display;
+	},
+
+
+	/**
+	 * Create the internal `columns` array with information about the columns
+	 * for the table. This includes determining which breakpoints the column
+	 * will appear in, based upon class names in the column, which makes up the
+	 * vast majority of this method.
+	 *
+	 * @private
+	 */
+	_classLogic: function ()
+	{
+		var that = this;
+		var calc = {};
+		var breakpoints = this.c.breakpoints;
+		var columns = this.s.dt.columns().eq(0).map( function (i) {
+			var className = this.column(i).header().className;
+
+			return {
+				className: className,
+				includeIn: [],
+				auto:      false,
+				control:   false,
+				never:     className.match(/\bnever\b/) ? true : false
+			};
+		} );
+
+		// Simply add a breakpoint to `includeIn` array, ensuring that there are
+		// no duplicates
+		var add = function ( colIdx, name ) {
+			var includeIn = columns[ colIdx ].includeIn;
+
+			if ( $.inArray( name, includeIn ) === -1 ) {
+				includeIn.push( name );
+			}
+		};
+
+		var column = function ( colIdx, name, operator, matched ) {
+			var size, i, ien;
+
+			if ( ! operator ) {
+				columns[ colIdx ].includeIn.push( name );
+			}
+			else if ( operator === 'max-' ) {
+				// Add this breakpoint and all smaller
+				size = that._find( name ).width;
+
+				for ( i=0, ien=breakpoints.length ; i<ien ; i++ ) {
+					if ( breakpoints[i].width <= size ) {
+						add( colIdx, breakpoints[i].name );
+					}
+				}
+			}
+			else if ( operator === 'min-' ) {
+				// Add this breakpoint and all larger
+				size = that._find( name ).width;
+
+				for ( i=0, ien=breakpoints.length ; i<ien ; i++ ) {
+					if ( breakpoints[i].width >= size ) {
+						add( colIdx, breakpoints[i].name );
+					}
+				}
+			}
+			else if ( operator === 'not-' ) {
+				// Add all but this breakpoint (xxx need extra information)
+
+				for ( i=0, ien=breakpoints.length ; i<ien ; i++ ) {
+					if ( breakpoints[i].name.indexOf( matched ) === -1 ) {
+						add( colIdx, breakpoints[i].name );
+					}
+				}
+			}
+		};
+
+		// Loop over each column and determine if it has a responsive control
+		// class
+		columns.each( function ( col, i ) {
+			var classNames = col.className.split(' ');
+			var hasClass = false;
+
+			// Split the class name up so multiple rules can be applied if needed
+			for ( var k=0, ken=classNames.length ; k<ken ; k++ ) {
+				var className = $.trim( classNames[k] );
+
+				if ( className === 'all' ) {
+					// Include in all
+					hasClass = true;
+					col.includeIn = $.map( breakpoints, function (a) {
+						return a.name;
+					} );
+					return;
+				}
+				else if ( className === 'none' || className === 'never' ) {
+					// Include in none (default) and no auto
+					hasClass = true;
+					return;
+				}
+				else if ( className === 'control' ) {
+					// Special column that is only visible, when one of the other
+					// columns is hidden. This is used for the details control
+					hasClass = true;
+					col.control = true;
+					return;
+				}
+
+				$.each( breakpoints, function ( j, breakpoint ) {
+					// Does this column have a class that matches this breakpoint?
+					var brokenPoint = breakpoint.name.split('-');
+					var re = new RegExp( '(min\\-|max\\-|not\\-)?('+brokenPoint[0]+')(\\-[_a-zA-Z0-9])?' );
+					var match = className.match( re );
+
+					if ( match ) {
+						hasClass = true;
+
+						if ( match[2] === brokenPoint[0] && match[3] === '-'+brokenPoint[1] ) {
+							// Class name matches breakpoint name fully
+							column( i, breakpoint.name, match[1], match[2]+match[3] );
+						}
+						else if ( match[2] === brokenPoint[0] && ! match[3] ) {
+							// Class name matched primary breakpoint name with no qualifier
+							column( i, breakpoint.name, match[1], match[2] );
+						}
+					}
+				} );
+			}
+
+			// If there was no control class, then automatic sizing is used
+			if ( ! hasClass ) {
+				col.auto = true;
+			}
+		} );
+
+		this.s.columns = columns;
+	},
+
+
+	/**
+	 * Initialisation for the details handler
+	 *
+	 * @private
+	 */
+	_detailsInit: function ()
+	{
+		var that    = this;
+		var dt      = this.s.dt;
+		var details = this.c.details;
+
+		// The inline type always uses the first child as the target
+		if ( details.type === 'inline' ) {
+			details.target = 'td:first-child';
+		}
+
+		// type.target can be a string jQuery selector or a column index
+		var target   = details.target;
+		var selector = typeof target === 'string' ? target : 'td';
+
+		// Click handler to show / hide the details rows when they are available
+		$( dt.table().body() ).on( 'click', selector, function (e) {
+			// If the table is not collapsed (i.e. there is no hidden columns)
+			// then take no action
+			if ( ! $(dt.table().node()).hasClass('collapsed' ) ) {
+				return;
+			}
+
+			// Check that the row is actually a DataTable's controlled node
+			if ( ! dt.row( $(this).closest('tr') ).length ) {
+				return;
+			}
+
+			// For column index, we determine if we should act or not in the
+			// handler - otherwise it is already okay
+			if ( typeof target === 'number' ) {
+				var targetIdx = target < 0 ?
+					dt.columns().eq(0).length + target :
+					target;
+
+				if ( dt.cell( this ).index().column !== targetIdx ) {
+					return;
+				}
+			}
+
+			// $().closest() includes itself in its check
+			var row = dt.row( $(this).closest('tr') );
+
+			if ( row.child.isShown() ) {
+				row.child( false );
+				$( row.node() ).removeClass( 'parent' );
+			}
+			else {
+				var info = that.c.details.renderer( dt, row[0] );
+				row.child( info, 'child' ).show();
+				$( row.node() ).addClass( 'parent' );
+			}
+		} );
+	},
+
+
+	/**
+	 * Update the child rows in the table whenever the column visibility changes
+	 *
+	 * @private
+	 */
+	_detailsVis: function ()
+	{
+		var that = this;
+		var dt = this.s.dt;
+
+		// Find how many columns are hidden
+		var hiddenColumns = dt.columns().indexes().filter( function ( idx ) {
+			var col = dt.column( idx );
+
+			if ( col.visible() ) {
+				return null;
+			}
+
+			// Only counts as hidden if it doesn't have the `never` class
+			return $( col.header() ).hasClass( 'never' ) ? null : idx;
+		} );
+		var haveHidden = true;
+
+		if ( hiddenColumns.length === 0 || ( hiddenColumns.length === 1 && this.s.columns[ hiddenColumns[0] ].control ) ) {
+			haveHidden = false;
+		}
+
+		if ( haveHidden ) {
+			// Show all existing child rows
+			dt.rows( { page: 'current' } ).eq(0).each( function (idx) {
+				var row = dt.row( idx );
+
+				if ( row.child() ) {
+					var info = that.c.details.renderer( dt, row[0] );
+
+					// The renderer can return false to have no child row
+					if ( info === false ) {
+						row.child.hide();
+					}
+					else {
+						row.child( info, 'child' ).show();
+					}
+				}
+			} );
+		}
+		else {
+			// Hide all existing child rows
+			dt.rows( { page: 'current' } ).eq(0).each( function (idx) {
+				dt.row( idx ).child.hide();
+			} );
+		}
+	},
+
+
+	/**
+	 * Find a breakpoint object from a name
+	 * @param  {string} name Breakpoint name to find
+	 * @return {object}      Breakpoint description object
+	 */
+	_find: function ( name )
+	{
+		var breakpoints = this.c.breakpoints;
+
+		for ( var i=0, ien=breakpoints.length ; i<ien ; i++ ) {
+			if ( breakpoints[i].name === name ) {
+				return breakpoints[i];
+			}
+		}
+	},
+
+
+	/**
+	 * Alter the table display for a resized viewport. This involves first
+	 * determining what breakpoint the window currently is in, getting the
+	 * column visibilities to apply and then setting them.
+	 *
+	 * @private
+	 */
+	_resize: function ()
+	{
+		var dt = this.s.dt;
+		var width = $(window).width();
+		var breakpoints = this.c.breakpoints;
+		var breakpoint = breakpoints[0].name;
+		var columns = this.s.columns;
+		var i, ien;
+
+		// Determine what breakpoint we are currently at
+		for ( i=breakpoints.length-1 ; i>=0 ; i-- ) {
+			if ( width <= breakpoints[i].width ) {
+				breakpoint = breakpoints[i].name;
+				break;
+			}
+		}
+		
+		// Show the columns for that break point
+		var columnsVis = this._columnsVisiblity( breakpoint );
+
+		// Set the class before the column visibility is changed so event
+		// listeners know what the state is. Need to determine if there are
+		// any columns that are not visible but can be shown
+		var collapsedClass = false;
+		for ( i=0, ien=columns.length ; i<ien ; i++ ) {
+			if ( columnsVis[i] === false && ! columns[i].never ) {
+				collapsedClass = true;
+				break;
+			}
+		}
+
+		$( dt.table().node() ).toggleClass('collapsed', collapsedClass );
+
+		dt.columns().eq(0).each( function ( colIdx, i ) {
+			dt.column( colIdx ).visible( columnsVis[i] );
+		} );
+	},
+
+
+	/**
+	 * Determine the width of each column in the table so the auto column hiding
+	 * has that information to work with. This method is never going to be 100%
+	 * perfect since column widths can change slightly per page, but without
+	 * seriously compromising performance this is quite effective.
+	 *
+	 * @private
+	 */
+	_resizeAuto: function ()
+	{
+		var dt = this.s.dt;
+		var columns = this.s.columns;
+
+		// Are we allowed to do auto sizing?
+		if ( ! this.c.auto ) {
+			return;
+		}
+
+		// Are there any columns that actually need auto-sizing, or do they all
+		// have classes defined
+		if ( $.inArray( true, $.map( columns, function (c) { return c.auto; } ) ) === -1 ) {
+			return;
+		}
+
+		// Clone the table with the current data in it
+		var tableWidth   = dt.table().node().offsetWidth;
+		var columnWidths = dt.columns;
+		var clonedTable  = dt.table().node().cloneNode( false );
+		var clonedHeader = $( dt.table().header().cloneNode( false ) ).appendTo( clonedTable );
+		var clonedBody   = $( dt.table().body().cloneNode( false ) ).appendTo( clonedTable );
+
+		$( dt.table().footer() ).clone( false ).appendTo( clonedTable );
+
+		// This is a bit slow, but we need to get a clone of each row that
+		// includes all columns. As such, try to do this as little as possible.
+		dt.rows( { page: 'current' } ).indexes().flatten().each( function ( idx ) {
+			var clone = dt.row( idx ).node().cloneNode( true );
+			
+			if ( dt.columns( ':hidden' ).flatten().length ) {
+				$(clone).append( dt.cells( idx, ':hidden' ).nodes().to$().clone() );
+			}
+
+			$(clone).appendTo( clonedBody );
+		} );
+
+		var cells = dt.columns().header().to$().clone( false );
+		$('<tr/>')
+			.append( cells )
+			.appendTo( clonedHeader );
+
+		// In the inline case extra padding is applied to the first column to
+		// give space for the show / hide icon. We need to use this in the
+		// calculation
+		if ( this.c.details.type === 'inline' ) {
+			$(clonedTable).addClass( 'dtr-inline collapsed' );
+		}
+
+		var inserted = $('<div/>')
+			.css( {
+				width: 1,
+				height: 1,
+				overflow: 'hidden'
+			} )
+			.append( clonedTable );
+
+		// Remove columns which are not to be included
+		inserted.find('th.never, td.never').remove();
+
+		inserted.insertBefore( dt.table().node() );
+
+		// The cloned header now contains the smallest that each column can be
+		dt.columns().eq(0).each( function ( idx ) {
+			columns[idx].minWidth = cells[ idx ].offsetWidth || 0;
+		} );
+
+		inserted.remove();
+	}
+};
+
+
+/**
+ * List of default breakpoints. Each item in the array is an object with two
+ * properties:
+ *
+ * * `name` - the breakpoint name.
+ * * `width` - the breakpoint width
+ *
+ * @name Responsive.breakpoints
+ * @static
+ */
+Responsive.breakpoints = [
+	{ name: 'desktop',  width: Infinity },
+	{ name: 'tablet-l', width: 1024 },
+	{ name: 'tablet-p', width: 768 },
+	{ name: 'mobile-l', width: 480 },
+	{ name: 'mobile-p', width: 320 }
+];
+
+
+/**
+ * Responsive default settings for initialisation
+ *
+ * @namespace
+ * @name Responsive.defaults
+ * @static
+ */
+Responsive.defaults = {
+	/**
+	 * List of breakpoints for the instance. Note that this means that each
+	 * instance can have its own breakpoints. Additionally, the breakpoints
+	 * cannot be changed once an instance has been creased.
+	 *
+	 * @type {Array}
+	 * @default Takes the value of `Responsive.breakpoints`
+	 */
+	breakpoints: Responsive.breakpoints,
+
+	/**
+	 * Enable / disable auto hiding calculations. It can help to increase
+	 * performance slightly if you disable this option, but all columns would
+	 * need to have breakpoint classes assigned to them
+	 *
+	 * @type {Boolean}
+	 * @default  `true`
+	 */
+	auto: true,
+
+	/**
+	 * Details control. If given as a string value, the `type` property of the
+	 * default object is set to that value, and the defaults used for the rest
+	 * of the object - this is for ease of implementation.
+	 *
+	 * The object consists of the following properties:
+	 *
+	 * * `renderer` - function that is called for display of the child row data.
+	 *   The default function will show the data from the hidden columns
+	 * * `target` - Used as the selector for what objects to attach the child
+	 *   open / close to
+	 * * `type` - `false` to disable the details display, `inline` or `column`
+	 *   for the two control types
+	 *
+	 * @type {Object|string}
+	 */
+	details: {
+		renderer: function ( api, rowIdx ) {
+			var data = api.cells( rowIdx, ':hidden' ).eq(0).map( function ( cell ) {
+				var header = $( api.column( cell.column ).header() );
+				var idx = api.cell( cell ).index();
+
+				if ( header.hasClass( 'control' ) || header.hasClass( 'never' ) ) {
+					return '';
+				}
+
+				// Use a non-public DT API method to render the data for display
+				// This needs to be updated when DT adds a suitable method for
+				// this type of data retrieval
+				var dtPrivate = api.settings()[0];
+				var cellData = dtPrivate.oApi._fnGetCellData(
+					dtPrivate, idx.row, idx.column, 'display'
+				);
+				var title = header.text();
+				if ( title ) {
+					title = title + ':';
+				}
+
+				return '<li data-dtr-index="'+idx.column+'">'+
+						'<span class="dtr-title">'+
+							title+
+						'</span> '+
+						'<span class="dtr-data">'+
+							cellData+
+						'</span>'+
+					'</li>';
+			} ).toArray().join('');
+
+			return data ?
+				$('<ul data-dtr-index="'+rowIdx+'"/>').append( data ) :
+				false;
+		},
+
+		target: 0,
+
+		type: 'inline'
+	}
+};
+
+
+/*
+ * API
+ */
+var Api = $.fn.dataTable.Api;
+
+// Doesn't do anything - work around for a bug in DT... Not documented
+Api.register( 'responsive()', function () {
+	return this;
+} );
+
+Api.register( 'responsive.index()', function ( li ) {
+	li = $(li);
+
+	return {
+		column: li.data('dtr-index'),
+		row:    li.parent().data('dtr-index')
+	};
+} );
+
+Api.register( 'responsive.rebuild()', function () {
+	return this.iterator( 'table', function ( ctx ) {
+		if ( ctx._responsive ) {
+			ctx._responsive._classLogic();
+		}
+	} );
+} );
+
+Api.register( 'responsive.recalc()', function () {
+	return this.iterator( 'table', function ( ctx ) {
+		if ( ctx._responsive ) {
+			ctx._responsive._resizeAuto();
+			ctx._responsive._resize();
+		}
+	} );
+} );
+
+
+/**
+ * Version information
+ *
+ * @name Responsive.version
+ * @static
+ */
+Responsive.version = '1.0.6';
+
+
+$.fn.dataTable.Responsive = Responsive;
+$.fn.DataTable.Responsive = Responsive;
+
+// Attach a listener to the document which listens for DataTables initialisation
+// events so we can automatically initialise
+$(document).on( 'init.dt.dtr', function (e, settings, json) {
+	if ( e.namespace !== 'dt' ) {
+		return;
+	}
+
+	if ( $(settings.nTable).hasClass( 'responsive' ) ||
+		 $(settings.nTable).hasClass( 'dt-responsive' ) ||
+		 settings.oInit.responsive ||
+		 DataTable.defaults.responsive
+	) {
+		var init = settings.oInit.responsive;
+
+		if ( init !== false ) {
+			new Responsive( settings, $.isPlainObject( init ) ? init : {}  );
+		}
+	}
+} );
+
+return Responsive;
+}; // /factory
+
+
+// Define as an AMD module if possible
+if ( typeof define === 'function' && define.amd ) {
+	define( ['jquery', 'datatables'], factory );
+}
+else if ( typeof exports === 'object' ) {
+    // Node/CommonJS
+    factory( require('jquery'), require('datatables') );
+}
+else if ( jQuery && !jQuery.fn.dataTable.Responsive ) {
+	// Otherwise simply initialise as normal, stopping multiple evaluation
+	factory( jQuery, jQuery.fn.dataTable );
+}
+
+
+})(window, document);

static/jquery.dataTables.min.js

@@ -0,0 +1,166 @@
+/*!
+ DataTables 1.10.12
+ ©2008-2015 SpryMedia Ltd - datatables.net/license
+*/
+(function(h){"function"===typeof define&&define.amd?define(["jquery"],function(D){return h(D,window,document)}):"object"===typeof exports?module.exports=function(D,I){D||(D=window);I||(I="undefined"!==typeof window?require("jquery"):require("jquery")(D));return h(I,D,D.document)}:h(jQuery,window,document)})(function(h,D,I,k){function X(a){var b,c,d={};h.each(a,function(e){if((b=e.match(/^([^A-Z]+?)([A-Z])/))&&-1!=="a aa ai ao as b fn i m o s ".indexOf(b[1]+" "))c=e.replace(b[0],b[2].toLowerCase()),
+d[c]=e,"o"===b[1]&&X(a[e])});a._hungarianMap=d}function K(a,b,c){a._hungarianMap||X(a);var d;h.each(b,function(e){d=a._hungarianMap[e];if(d!==k&&(c||b[d]===k))"o"===d.charAt(0)?(b[d]||(b[d]={}),h.extend(!0,b[d],b[e]),K(a[d],b[d],c)):b[d]=b[e]})}function Da(a){var b=m.defaults.oLanguage,c=a.sZeroRecords;!a.sEmptyTable&&(c&&"No data available in table"===b.sEmptyTable)&&E(a,a,"sZeroRecords","sEmptyTable");!a.sLoadingRecords&&(c&&"Loading..."===b.sLoadingRecords)&&E(a,a,"sZeroRecords","sLoadingRecords");
+a.sInfoThousands&&(a.sThousands=a.sInfoThousands);(a=a.sDecimal)&&db(a)}function eb(a){A(a,"ordering","bSort");A(a,"orderMulti","bSortMulti");A(a,"orderClasses","bSortClasses");A(a,"orderCellsTop","bSortCellsTop");A(a,"order","aaSorting");A(a,"orderFixed","aaSortingFixed");A(a,"paging","bPaginate");A(a,"pagingType","sPaginationType");A(a,"pageLength","iDisplayLength");A(a,"searching","bFilter");"boolean"===typeof a.sScrollX&&(a.sScrollX=a.sScrollX?"100%":"");"boolean"===typeof a.scrollX&&(a.scrollX=
+a.scrollX?"100%":"");if(a=a.aoSearchCols)for(var b=0,c=a.length;b<c;b++)a[b]&&K(m.models.oSearch,a[b])}function fb(a){A(a,"orderable","bSortable");A(a,"orderData","aDataSort");A(a,"orderSequence","asSorting");A(a,"orderDataType","sortDataType");var b=a.aDataSort;b&&!h.isArray(b)&&(a.aDataSort=[b])}function gb(a){if(!m.__browser){var b={};m.__browser=b;var c=h("<div/>").css({position:"fixed",top:0,left:0,height:1,width:1,overflow:"hidden"}).append(h("<div/>").css({position:"absolute",top:1,left:1,
+width:100,overflow:"scroll"}).append(h("<div/>").css({width:"100%",height:10}))).appendTo("body"),d=c.children(),e=d.children();b.barWidth=d[0].offsetWidth-d[0].clientWidth;b.bScrollOversize=100===e[0].offsetWidth&&100!==d[0].clientWidth;b.bScrollbarLeft=1!==Math.round(e.offset().left);b.bBounding=c[0].getBoundingClientRect().width?!0:!1;c.remove()}h.extend(a.oBrowser,m.__browser);a.oScroll.iBarWidth=m.__browser.barWidth}function hb(a,b,c,d,e,f){var g,j=!1;c!==k&&(g=c,j=!0);for(;d!==e;)a.hasOwnProperty(d)&&
+(g=j?b(g,a[d],d,a):a[d],j=!0,d+=f);return g}function Ea(a,b){var c=m.defaults.column,d=a.aoColumns.length,c=h.extend({},m.models.oColumn,c,{nTh:b?b:I.createElement("th"),sTitle:c.sTitle?c.sTitle:b?b.innerHTML:"",aDataSort:c.aDataSort?c.aDataSort:[d],mData:c.mData?c.mData:d,idx:d});a.aoColumns.push(c);c=a.aoPreSearchCols;c[d]=h.extend({},m.models.oSearch,c[d]);ja(a,d,h(b).data())}function ja(a,b,c){var b=a.aoColumns[b],d=a.oClasses,e=h(b.nTh);if(!b.sWidthOrig){b.sWidthOrig=e.attr("width")||null;var f=
+(e.attr("style")||"").match(/width:\s*(\d+[pxem%]+)/);f&&(b.sWidthOrig=f[1])}c!==k&&null!==c&&(fb(c),K(m.defaults.column,c),c.mDataProp!==k&&!c.mData&&(c.mData=c.mDataProp),c.sType&&(b._sManualType=c.sType),c.className&&!c.sClass&&(c.sClass=c.className),h.extend(b,c),E(b,c,"sWidth","sWidthOrig"),c.iDataSort!==k&&(b.aDataSort=[c.iDataSort]),E(b,c,"aDataSort"));var g=b.mData,j=Q(g),i=b.mRender?Q(b.mRender):null,c=function(a){return"string"===typeof a&&-1!==a.indexOf("@")};b._bAttrSrc=h.isPlainObject(g)&&
+(c(g.sort)||c(g.type)||c(g.filter));b._setter=null;b.fnGetData=function(a,b,c){var d=j(a,b,k,c);return i&&b?i(d,b,a,c):d};b.fnSetData=function(a,b,c){return R(g)(a,b,c)};"number"!==typeof g&&(a._rowReadObject=!0);a.oFeatures.bSort||(b.bSortable=!1,e.addClass(d.sSortableNone));a=-1!==h.inArray("asc",b.asSorting);c=-1!==h.inArray("desc",b.asSorting);!b.bSortable||!a&&!c?(b.sSortingClass=d.sSortableNone,b.sSortingClassJUI=""):a&&!c?(b.sSortingClass=d.sSortableAsc,b.sSortingClassJUI=d.sSortJUIAscAllowed):
+!a&&c?(b.sSortingClass=d.sSortableDesc,b.sSortingClassJUI=d.sSortJUIDescAllowed):(b.sSortingClass=d.sSortable,b.sSortingClassJUI=d.sSortJUI)}function Y(a){if(!1!==a.oFeatures.bAutoWidth){var b=a.aoColumns;Fa(a);for(var c=0,d=b.length;c<d;c++)b[c].nTh.style.width=b[c].sWidth}b=a.oScroll;(""!==b.sY||""!==b.sX)&&ka(a);u(a,null,"column-sizing",[a])}function Z(a,b){var c=la(a,"bVisible");return"number"===typeof c[b]?c[b]:null}function $(a,b){var c=la(a,"bVisible"),c=h.inArray(b,c);return-1!==c?c:null}
+function aa(a){var b=0;h.each(a.aoColumns,function(a,d){d.bVisible&&"none"!==h(d.nTh).css("display")&&b++});return b}function la(a,b){var c=[];h.map(a.aoColumns,function(a,e){a[b]&&c.push(e)});return c}function Ga(a){var b=a.aoColumns,c=a.aoData,d=m.ext.type.detect,e,f,g,j,i,h,l,q,t;e=0;for(f=b.length;e<f;e++)if(l=b[e],t=[],!l.sType&&l._sManualType)l.sType=l._sManualType;else if(!l.sType){g=0;for(j=d.length;g<j;g++){i=0;for(h=c.length;i<h;i++){t[i]===k&&(t[i]=B(a,i,e,"type"));q=d[g](t[i],a);if(!q&&
+g!==d.length-1)break;if("html"===q)break}if(q){l.sType=q;break}}l.sType||(l.sType="string")}}function ib(a,b,c,d){var e,f,g,j,i,n,l=a.aoColumns;if(b)for(e=b.length-1;0<=e;e--){n=b[e];var q=n.targets!==k?n.targets:n.aTargets;h.isArray(q)||(q=[q]);f=0;for(g=q.length;f<g;f++)if("number"===typeof q[f]&&0<=q[f]){for(;l.length<=q[f];)Ea(a);d(q[f],n)}else if("number"===typeof q[f]&&0>q[f])d(l.length+q[f],n);else if("string"===typeof q[f]){j=0;for(i=l.length;j<i;j++)("_all"==q[f]||h(l[j].nTh).hasClass(q[f]))&&
+d(j,n)}}if(c){e=0;for(a=c.length;e<a;e++)d(e,c[e])}}function N(a,b,c,d){var e=a.aoData.length,f=h.extend(!0,{},m.models.oRow,{src:c?"dom":"data",idx:e});f._aData=b;a.aoData.push(f);for(var g=a.aoColumns,j=0,i=g.length;j<i;j++)g[j].sType=null;a.aiDisplayMaster.push(e);b=a.rowIdFn(b);b!==k&&(a.aIds[b]=f);(c||!a.oFeatures.bDeferRender)&&Ha(a,e,c,d);return e}function ma(a,b){var c;b instanceof h||(b=h(b));return b.map(function(b,e){c=Ia(a,e);return N(a,c.data,e,c.cells)})}function B(a,b,c,d){var e=a.iDraw,
+f=a.aoColumns[c],g=a.aoData[b]._aData,j=f.sDefaultContent,i=f.fnGetData(g,d,{settings:a,row:b,col:c});if(i===k)return a.iDrawError!=e&&null===j&&(L(a,0,"Requested unknown parameter "+("function"==typeof f.mData?"{function}":"'"+f.mData+"'")+" for row "+b+", column "+c,4),a.iDrawError=e),j;if((i===g||null===i)&&null!==j&&d!==k)i=j;else if("function"===typeof i)return i.call(g);return null===i&&"display"==d?"":i}function jb(a,b,c,d){a.aoColumns[c].fnSetData(a.aoData[b]._aData,d,{settings:a,row:b,col:c})}
+function Ja(a){return h.map(a.match(/(\\.|[^\.])+/g)||[""],function(a){return a.replace(/\\./g,".")})}function Q(a){if(h.isPlainObject(a)){var b={};h.each(a,function(a,c){c&&(b[a]=Q(c))});return function(a,c,f,g){var j=b[c]||b._;return j!==k?j(a,c,f,g):a}}if(null===a)return function(a){return a};if("function"===typeof a)return function(b,c,f,g){return a(b,c,f,g)};if("string"===typeof a&&(-1!==a.indexOf(".")||-1!==a.indexOf("[")||-1!==a.indexOf("("))){var c=function(a,b,f){var g,j;if(""!==f){j=Ja(f);
+for(var i=0,n=j.length;i<n;i++){f=j[i].match(ba);g=j[i].match(U);if(f){j[i]=j[i].replace(ba,"");""!==j[i]&&(a=a[j[i]]);g=[];j.splice(0,i+1);j=j.join(".");if(h.isArray(a)){i=0;for(n=a.length;i<n;i++)g.push(c(a[i],b,j))}a=f[0].substring(1,f[0].length-1);a=""===a?g:g.join(a);break}else if(g){j[i]=j[i].replace(U,"");a=a[j[i]]();continue}if(null===a||a[j[i]]===k)return k;a=a[j[i]]}}return a};return function(b,e){return c(b,e,a)}}return function(b){return b[a]}}function R(a){if(h.isPlainObject(a))return R(a._);
+if(null===a)return function(){};if("function"===typeof a)return function(b,d,e){a(b,"set",d,e)};if("string"===typeof a&&(-1!==a.indexOf(".")||-1!==a.indexOf("[")||-1!==a.indexOf("("))){var b=function(a,d,e){var e=Ja(e),f;f=e[e.length-1];for(var g,j,i=0,n=e.length-1;i<n;i++){g=e[i].match(ba);j=e[i].match(U);if(g){e[i]=e[i].replace(ba,"");a[e[i]]=[];f=e.slice();f.splice(0,i+1);g=f.join(".");if(h.isArray(d)){j=0;for(n=d.length;j<n;j++)f={},b(f,d[j],g),a[e[i]].push(f)}else a[e[i]]=d;return}j&&(e[i]=e[i].replace(U,
+""),a=a[e[i]](d));if(null===a[e[i]]||a[e[i]]===k)a[e[i]]={};a=a[e[i]]}if(f.match(U))a[f.replace(U,"")](d);else a[f.replace(ba,"")]=d};return function(c,d){return b(c,d,a)}}return function(b,d){b[a]=d}}function Ka(a){return G(a.aoData,"_aData")}function na(a){a.aoData.length=0;a.aiDisplayMaster.length=0;a.aiDisplay.length=0;a.aIds={}}function oa(a,b,c){for(var d=-1,e=0,f=a.length;e<f;e++)a[e]==b?d=e:a[e]>b&&a[e]--; -1!=d&&c===k&&a.splice(d,1)}function ca(a,b,c,d){var e=a.aoData[b],f,g=function(c,d){for(;c.childNodes.length;)c.removeChild(c.firstChild);
+c.innerHTML=B(a,b,d,"display")};if("dom"===c||(!c||"auto"===c)&&"dom"===e.src)e._aData=Ia(a,e,d,d===k?k:e._aData).data;else{var j=e.anCells;if(j)if(d!==k)g(j[d],d);else{c=0;for(f=j.length;c<f;c++)g(j[c],c)}}e._aSortData=null;e._aFilterData=null;g=a.aoColumns;if(d!==k)g[d].sType=null;else{c=0;for(f=g.length;c<f;c++)g[c].sType=null;La(a,e)}}function Ia(a,b,c,d){var e=[],f=b.firstChild,g,j,i=0,n,l=a.aoColumns,q=a._rowReadObject,d=d!==k?d:q?{}:[],t=function(a,b){if("string"===typeof a){var c=a.indexOf("@");
+-1!==c&&(c=a.substring(c+1),R(a)(d,b.getAttribute(c)))}},S=function(a){if(c===k||c===i)j=l[i],n=h.trim(a.innerHTML),j&&j._bAttrSrc?(R(j.mData._)(d,n),t(j.mData.sort,a),t(j.mData.type,a),t(j.mData.filter,a)):q?(j._setter||(j._setter=R(j.mData)),j._setter(d,n)):d[i]=n;i++};if(f)for(;f;){g=f.nodeName.toUpperCase();if("TD"==g||"TH"==g)S(f),e.push(f);f=f.nextSibling}else{e=b.anCells;f=0;for(g=e.length;f<g;f++)S(e[f])}if(b=b.firstChild?b:b.nTr)(b=b.getAttribute("id"))&&R(a.rowId)(d,b);return{data:d,cells:e}}
+function Ha(a,b,c,d){var e=a.aoData[b],f=e._aData,g=[],j,i,n,l,q;if(null===e.nTr){j=c||I.createElement("tr");e.nTr=j;e.anCells=g;j._DT_RowIndex=b;La(a,e);l=0;for(q=a.aoColumns.length;l<q;l++){n=a.aoColumns[l];i=c?d[l]:I.createElement(n.sCellType);i._DT_CellIndex={row:b,column:l};g.push(i);if((!c||n.mRender||n.mData!==l)&&(!h.isPlainObject(n.mData)||n.mData._!==l+".display"))i.innerHTML=B(a,b,l,"display");n.sClass&&(i.className+=" "+n.sClass);n.bVisible&&!c?j.appendChild(i):!n.bVisible&&c&&i.parentNode.removeChild(i);
+n.fnCreatedCell&&n.fnCreatedCell.call(a.oInstance,i,B(a,b,l),f,b,l)}u(a,"aoRowCreatedCallback",null,[j,f,b])}e.nTr.setAttribute("role","row")}function La(a,b){var c=b.nTr,d=b._aData;if(c){var e=a.rowIdFn(d);e&&(c.id=e);d.DT_RowClass&&(e=d.DT_RowClass.split(" "),b.__rowc=b.__rowc?pa(b.__rowc.concat(e)):e,h(c).removeClass(b.__rowc.join(" ")).addClass(d.DT_RowClass));d.DT_RowAttr&&h(c).attr(d.DT_RowAttr);d.DT_RowData&&h(c).data(d.DT_RowData)}}function kb(a){var b,c,d,e,f,g=a.nTHead,j=a.nTFoot,i=0===
+h("th, td",g).length,n=a.oClasses,l=a.aoColumns;i&&(e=h("<tr/>").appendTo(g));b=0;for(c=l.length;b<c;b++)f=l[b],d=h(f.nTh).addClass(f.sClass),i&&d.appendTo(e),a.oFeatures.bSort&&(d.addClass(f.sSortingClass),!1!==f.bSortable&&(d.attr("tabindex",a.iTabIndex).attr("aria-controls",a.sTableId),Ma(a,f.nTh,b))),f.sTitle!=d[0].innerHTML&&d.html(f.sTitle),Na(a,"header")(a,d,f,n);i&&da(a.aoHeader,g);h(g).find(">tr").attr("role","row");h(g).find(">tr>th, >tr>td").addClass(n.sHeaderTH);h(j).find(">tr>th, >tr>td").addClass(n.sFooterTH);
+if(null!==j){a=a.aoFooter[0];b=0;for(c=a.length;b<c;b++)f=l[b],f.nTf=a[b].cell,f.sClass&&h(f.nTf).addClass(f.sClass)}}function ea(a,b,c){var d,e,f,g=[],j=[],i=a.aoColumns.length,n;if(b){c===k&&(c=!1);d=0;for(e=b.length;d<e;d++){g[d]=b[d].slice();g[d].nTr=b[d].nTr;for(f=i-1;0<=f;f--)!a.aoColumns[f].bVisible&&!c&&g[d].splice(f,1);j.push([])}d=0;for(e=g.length;d<e;d++){if(a=g[d].nTr)for(;f=a.firstChild;)a.removeChild(f);f=0;for(b=g[d].length;f<b;f++)if(n=i=1,j[d][f]===k){a.appendChild(g[d][f].cell);
+for(j[d][f]=1;g[d+i]!==k&&g[d][f].cell==g[d+i][f].cell;)j[d+i][f]=1,i++;for(;g[d][f+n]!==k&&g[d][f].cell==g[d][f+n].cell;){for(c=0;c<i;c++)j[d+c][f+n]=1;n++}h(g[d][f].cell).attr("rowspan",i).attr("colspan",n)}}}}function O(a){var b=u(a,"aoPreDrawCallback","preDraw",[a]);if(-1!==h.inArray(!1,b))C(a,!1);else{var b=[],c=0,d=a.asStripeClasses,e=d.length,f=a.oLanguage,g=a.iInitDisplayStart,j="ssp"==y(a),i=a.aiDisplay;a.bDrawing=!0;g!==k&&-1!==g&&(a._iDisplayStart=j?g:g>=a.fnRecordsDisplay()?0:g,a.iInitDisplayStart=
+-1);var g=a._iDisplayStart,n=a.fnDisplayEnd();if(a.bDeferLoading)a.bDeferLoading=!1,a.iDraw++,C(a,!1);else if(j){if(!a.bDestroying&&!lb(a))return}else a.iDraw++;if(0!==i.length){f=j?a.aoData.length:n;for(j=j?0:g;j<f;j++){var l=i[j],q=a.aoData[l];null===q.nTr&&Ha(a,l);l=q.nTr;if(0!==e){var t=d[c%e];q._sRowStripe!=t&&(h(l).removeClass(q._sRowStripe).addClass(t),q._sRowStripe=t)}u(a,"aoRowCallback",null,[l,q._aData,c,j]);b.push(l);c++}}else c=f.sZeroRecords,1==a.iDraw&&"ajax"==y(a)?c=f.sLoadingRecords:
+f.sEmptyTable&&0===a.fnRecordsTotal()&&(c=f.sEmptyTable),b[0]=h("<tr/>",{"class":e?d[0]:""}).append(h("<td />",{valign:"top",colSpan:aa(a),"class":a.oClasses.sRowEmpty}).html(c))[0];u(a,"aoHeaderCallback","header",[h(a.nTHead).children("tr")[0],Ka(a),g,n,i]);u(a,"aoFooterCallback","footer",[h(a.nTFoot).children("tr")[0],Ka(a),g,n,i]);d=h(a.nTBody);d.children().detach();d.append(h(b));u(a,"aoDrawCallback","draw",[a]);a.bSorted=!1;a.bFiltered=!1;a.bDrawing=!1}}function T(a,b){var c=a.oFeatures,d=c.bFilter;
+c.bSort&&mb(a);d?fa(a,a.oPreviousSearch):a.aiDisplay=a.aiDisplayMaster.slice();!0!==b&&(a._iDisplayStart=0);a._drawHold=b;O(a);a._drawHold=!1}function nb(a){var b=a.oClasses,c=h(a.nTable),c=h("<div/>").insertBefore(c),d=a.oFeatures,e=h("<div/>",{id:a.sTableId+"_wrapper","class":b.sWrapper+(a.nTFoot?"":" "+b.sNoFooter)});a.nHolding=c[0];a.nTableWrapper=e[0];a.nTableReinsertBefore=a.nTable.nextSibling;for(var f=a.sDom.split(""),g,j,i,n,l,q,t=0;t<f.length;t++){g=null;j=f[t];if("<"==j){i=h("<div/>")[0];
+n=f[t+1];if("'"==n||'"'==n){l="";for(q=2;f[t+q]!=n;)l+=f[t+q],q++;"H"==l?l=b.sJUIHeader:"F"==l&&(l=b.sJUIFooter);-1!=l.indexOf(".")?(n=l.split("."),i.id=n[0].substr(1,n[0].length-1),i.className=n[1]):"#"==l.charAt(0)?i.id=l.substr(1,l.length-1):i.className=l;t+=q}e.append(i);e=h(i)}else if(">"==j)e=e.parent();else if("l"==j&&d.bPaginate&&d.bLengthChange)g=ob(a);else if("f"==j&&d.bFilter)g=pb(a);else if("r"==j&&d.bProcessing)g=qb(a);else if("t"==j)g=rb(a);else if("i"==j&&d.bInfo)g=sb(a);else if("p"==
+j&&d.bPaginate)g=tb(a);else if(0!==m.ext.feature.length){i=m.ext.feature;q=0;for(n=i.length;q<n;q++)if(j==i[q].cFeature){g=i[q].fnInit(a);break}}g&&(i=a.aanFeatures,i[j]||(i[j]=[]),i[j].push(g),e.append(g))}c.replaceWith(e);a.nHolding=null}function da(a,b){var c=h(b).children("tr"),d,e,f,g,j,i,n,l,q,t;a.splice(0,a.length);f=0;for(i=c.length;f<i;f++)a.push([]);f=0;for(i=c.length;f<i;f++){d=c[f];for(e=d.firstChild;e;){if("TD"==e.nodeName.toUpperCase()||"TH"==e.nodeName.toUpperCase()){l=1*e.getAttribute("colspan");
+q=1*e.getAttribute("rowspan");l=!l||0===l||1===l?1:l;q=!q||0===q||1===q?1:q;g=0;for(j=a[f];j[g];)g++;n=g;t=1===l?!0:!1;for(j=0;j<l;j++)for(g=0;g<q;g++)a[f+g][n+j]={cell:e,unique:t},a[f+g].nTr=d}e=e.nextSibling}}}function qa(a,b,c){var d=[];c||(c=a.aoHeader,b&&(c=[],da(c,b)));for(var b=0,e=c.length;b<e;b++)for(var f=0,g=c[b].length;f<g;f++)if(c[b][f].unique&&(!d[f]||!a.bSortCellsTop))d[f]=c[b][f].cell;return d}function ra(a,b,c){u(a,"aoServerParams","serverParams",[b]);if(b&&h.isArray(b)){var d={},
+e=/(.*?)\[\]$/;h.each(b,function(a,b){var c=b.name.match(e);c?(c=c[0],d[c]||(d[c]=[]),d[c].push(b.value)):d[b.name]=b.value});b=d}var f,g=a.ajax,j=a.oInstance,i=function(b){u(a,null,"xhr",[a,b,a.jqXHR]);c(b)};if(h.isPlainObject(g)&&g.data){f=g.data;var n=h.isFunction(f)?f(b,a):f,b=h.isFunction(f)&&n?n:h.extend(!0,b,n);delete g.data}n={data:b,success:function(b){var c=b.error||b.sError;c&&L(a,0,c);a.json=b;i(b)},dataType:"json",cache:!1,type:a.sServerMethod,error:function(b,c){var d=u(a,null,"xhr",
+[a,null,a.jqXHR]);-1===h.inArray(!0,d)&&("parsererror"==c?L(a,0,"Invalid JSON response",1):4===b.readyState&&L(a,0,"Ajax error",7));C(a,!1)}};a.oAjaxData=b;u(a,null,"preXhr",[a,b]);a.fnServerData?a.fnServerData.call(j,a.sAjaxSource,h.map(b,function(a,b){return{name:b,value:a}}),i,a):a.sAjaxSource||"string"===typeof g?a.jqXHR=h.ajax(h.extend(n,{url:g||a.sAjaxSource})):h.isFunction(g)?a.jqXHR=g.call(j,b,i,a):(a.jqXHR=h.ajax(h.extend(n,g)),g.data=f)}function lb(a){return a.bAjaxDataGet?(a.iDraw++,C(a,
+!0),ra(a,ub(a),function(b){vb(a,b)}),!1):!0}function ub(a){var b=a.aoColumns,c=b.length,d=a.oFeatures,e=a.oPreviousSearch,f=a.aoPreSearchCols,g,j=[],i,n,l,q=V(a);g=a._iDisplayStart;i=!1!==d.bPaginate?a._iDisplayLength:-1;var k=function(a,b){j.push({name:a,value:b})};k("sEcho",a.iDraw);k("iColumns",c);k("sColumns",G(b,"sName").join(","));k("iDisplayStart",g);k("iDisplayLength",i);var S={draw:a.iDraw,columns:[],order:[],start:g,length:i,search:{value:e.sSearch,regex:e.bRegex}};for(g=0;g<c;g++)n=b[g],
+l=f[g],i="function"==typeof n.mData?"function":n.mData,S.columns.push({data:i,name:n.sName,searchable:n.bSearchable,orderable:n.bSortable,search:{value:l.sSearch,regex:l.bRegex}}),k("mDataProp_"+g,i),d.bFilter&&(k("sSearch_"+g,l.sSearch),k("bRegex_"+g,l.bRegex),k("bSearchable_"+g,n.bSearchable)),d.bSort&&k("bSortable_"+g,n.bSortable);d.bFilter&&(k("sSearch",e.sSearch),k("bRegex",e.bRegex));d.bSort&&(h.each(q,function(a,b){S.order.push({column:b.col,dir:b.dir});k("iSortCol_"+a,b.col);k("sSortDir_"+
+a,b.dir)}),k("iSortingCols",q.length));b=m.ext.legacy.ajax;return null===b?a.sAjaxSource?j:S:b?j:S}function vb(a,b){var c=sa(a,b),d=b.sEcho!==k?b.sEcho:b.draw,e=b.iTotalRecords!==k?b.iTotalRecords:b.recordsTotal,f=b.iTotalDisplayRecords!==k?b.iTotalDisplayRecords:b.recordsFiltered;if(d){if(1*d<a.iDraw)return;a.iDraw=1*d}na(a);a._iRecordsTotal=parseInt(e,10);a._iRecordsDisplay=parseInt(f,10);d=0;for(e=c.length;d<e;d++)N(a,c[d]);a.aiDisplay=a.aiDisplayMaster.slice();a.bAjaxDataGet=!1;O(a);a._bInitComplete||
+ta(a,b);a.bAjaxDataGet=!0;C(a,!1)}function sa(a,b){var c=h.isPlainObject(a.ajax)&&a.ajax.dataSrc!==k?a.ajax.dataSrc:a.sAjaxDataProp;return"data"===c?b.aaData||b[c]:""!==c?Q(c)(b):b}function pb(a){var b=a.oClasses,c=a.sTableId,d=a.oLanguage,e=a.oPreviousSearch,f=a.aanFeatures,g='<input type="search" class="'+b.sFilterInput+'"/>',j=d.sSearch,j=j.match(/_INPUT_/)?j.replace("_INPUT_",g):j+g,b=h("<div/>",{id:!f.f?c+"_filter":null,"class":b.sFilter}).append(h("<label/>").append(j)),f=function(){var b=!this.value?
+"":this.value;b!=e.sSearch&&(fa(a,{sSearch:b,bRegex:e.bRegex,bSmart:e.bSmart,bCaseInsensitive:e.bCaseInsensitive}),a._iDisplayStart=0,O(a))},g=null!==a.searchDelay?a.searchDelay:"ssp"===y(a)?400:0,i=h("input",b).val(e.sSearch).attr("placeholder",d.sSearchPlaceholder).bind("keyup.DT search.DT input.DT paste.DT cut.DT",g?Oa(f,g):f).bind("keypress.DT",function(a){if(13==a.keyCode)return!1}).attr("aria-controls",c);h(a.nTable).on("search.dt.DT",function(b,c){if(a===c)try{i[0]!==I.activeElement&&i.val(e.sSearch)}catch(d){}});
+return b[0]}function fa(a,b,c){var d=a.oPreviousSearch,e=a.aoPreSearchCols,f=function(a){d.sSearch=a.sSearch;d.bRegex=a.bRegex;d.bSmart=a.bSmart;d.bCaseInsensitive=a.bCaseInsensitive};Ga(a);if("ssp"!=y(a)){wb(a,b.sSearch,c,b.bEscapeRegex!==k?!b.bEscapeRegex:b.bRegex,b.bSmart,b.bCaseInsensitive);f(b);for(b=0;b<e.length;b++)xb(a,e[b].sSearch,b,e[b].bEscapeRegex!==k?!e[b].bEscapeRegex:e[b].bRegex,e[b].bSmart,e[b].bCaseInsensitive);yb(a)}else f(b);a.bFiltered=!0;u(a,null,"search",[a])}function yb(a){for(var b=
+m.ext.search,c=a.aiDisplay,d,e,f=0,g=b.length;f<g;f++){for(var j=[],i=0,n=c.length;i<n;i++)e=c[i],d=a.aoData[e],b[f](a,d._aFilterData,e,d._aData,i)&&j.push(e);c.length=0;h.merge(c,j)}}function xb(a,b,c,d,e,f){if(""!==b)for(var g=a.aiDisplay,d=Pa(b,d,e,f),e=g.length-1;0<=e;e--)b=a.aoData[g[e]]._aFilterData[c],d.test(b)||g.splice(e,1)}function wb(a,b,c,d,e,f){var d=Pa(b,d,e,f),e=a.oPreviousSearch.sSearch,f=a.aiDisplayMaster,g;0!==m.ext.search.length&&(c=!0);g=zb(a);if(0>=b.length)a.aiDisplay=f.slice();
+else{if(g||c||e.length>b.length||0!==b.indexOf(e)||a.bSorted)a.aiDisplay=f.slice();b=a.aiDisplay;for(c=b.length-1;0<=c;c--)d.test(a.aoData[b[c]]._sFilterRow)||b.splice(c,1)}}function Pa(a,b,c,d){a=b?a:Qa(a);c&&(a="^(?=.*?"+h.map(a.match(/"[^"]+"|[^ ]+/g)||[""],function(a){if('"'===a.charAt(0))var b=a.match(/^"(.*)"$/),a=b?b[1]:a;return a.replace('"',"")}).join(")(?=.*?")+").*$");return RegExp(a,d?"i":"")}function zb(a){var b=a.aoColumns,c,d,e,f,g,j,i,h,l=m.ext.type.search;c=!1;d=0;for(f=a.aoData.length;d<
+f;d++)if(h=a.aoData[d],!h._aFilterData){j=[];e=0;for(g=b.length;e<g;e++)c=b[e],c.bSearchable?(i=B(a,d,e,"filter"),l[c.sType]&&(i=l[c.sType](i)),null===i&&(i=""),"string"!==typeof i&&i.toString&&(i=i.toString())):i="",i.indexOf&&-1!==i.indexOf("&")&&(ua.innerHTML=i,i=Zb?ua.textContent:ua.innerText),i.replace&&(i=i.replace(/[\r\n]/g,"")),j.push(i);h._aFilterData=j;h._sFilterRow=j.join("  ");c=!0}return c}function Ab(a){return{search:a.sSearch,smart:a.bSmart,regex:a.bRegex,caseInsensitive:a.bCaseInsensitive}}
+function Bb(a){return{sSearch:a.search,bSmart:a.smart,bRegex:a.regex,bCaseInsensitive:a.caseInsensitive}}function sb(a){var b=a.sTableId,c=a.aanFeatures.i,d=h("<div/>",{"class":a.oClasses.sInfo,id:!c?b+"_info":null});c||(a.aoDrawCallback.push({fn:Cb,sName:"information"}),d.attr("role","status").attr("aria-live","polite"),h(a.nTable).attr("aria-describedby",b+"_info"));return d[0]}function Cb(a){var b=a.aanFeatures.i;if(0!==b.length){var c=a.oLanguage,d=a._iDisplayStart+1,e=a.fnDisplayEnd(),f=a.fnRecordsTotal(),
+g=a.fnRecordsDisplay(),j=g?c.sInfo:c.sInfoEmpty;g!==f&&(j+=" "+c.sInfoFiltered);j+=c.sInfoPostFix;j=Db(a,j);c=c.fnInfoCallback;null!==c&&(j=c.call(a.oInstance,a,d,e,f,g,j));h(b).html(j)}}function Db(a,b){var c=a.fnFormatNumber,d=a._iDisplayStart+1,e=a._iDisplayLength,f=a.fnRecordsDisplay(),g=-1===e;return b.replace(/_START_/g,c.call(a,d)).replace(/_END_/g,c.call(a,a.fnDisplayEnd())).replace(/_MAX_/g,c.call(a,a.fnRecordsTotal())).replace(/_TOTAL_/g,c.call(a,f)).replace(/_PAGE_/g,c.call(a,g?1:Math.ceil(d/
+e))).replace(/_PAGES_/g,c.call(a,g?1:Math.ceil(f/e)))}function ga(a){var b,c,d=a.iInitDisplayStart,e=a.aoColumns,f;c=a.oFeatures;var g=a.bDeferLoading;if(a.bInitialised){nb(a);kb(a);ea(a,a.aoHeader);ea(a,a.aoFooter);C(a,!0);c.bAutoWidth&&Fa(a);b=0;for(c=e.length;b<c;b++)f=e[b],f.sWidth&&(f.nTh.style.width=x(f.sWidth));u(a,null,"preInit",[a]);T(a);e=y(a);if("ssp"!=e||g)"ajax"==e?ra(a,[],function(c){var f=sa(a,c);for(b=0;b<f.length;b++)N(a,f[b]);a.iInitDisplayStart=d;T(a);C(a,!1);ta(a,c)},a):(C(a,!1),
+ta(a))}else setTimeout(function(){ga(a)},200)}function ta(a,b){a._bInitComplete=!0;(b||a.oInit.aaData)&&Y(a);u(a,null,"plugin-init",[a,b]);u(a,"aoInitComplete","init",[a,b])}function Ra(a,b){var c=parseInt(b,10);a._iDisplayLength=c;Sa(a);u(a,null,"length",[a,c])}function ob(a){for(var b=a.oClasses,c=a.sTableId,d=a.aLengthMenu,e=h.isArray(d[0]),f=e?d[0]:d,d=e?d[1]:d,e=h("<select/>",{name:c+"_length","aria-controls":c,"class":b.sLengthSelect}),g=0,j=f.length;g<j;g++)e[0][g]=new Option(d[g],f[g]);var i=
+h("<div><label/></div>").addClass(b.sLength);a.aanFeatures.l||(i[0].id=c+"_length");i.children().append(a.oLanguage.sLengthMenu.replace("_MENU_",e[0].outerHTML));h("select",i).val(a._iDisplayLength).bind("change.DT",function(){Ra(a,h(this).val());O(a)});h(a.nTable).bind("length.dt.DT",function(b,c,d){a===c&&h("select",i).val(d)});return i[0]}function tb(a){var b=a.sPaginationType,c=m.ext.pager[b],d="function"===typeof c,e=function(a){O(a)},b=h("<div/>").addClass(a.oClasses.sPaging+b)[0],f=a.aanFeatures;
+d||c.fnInit(a,b,e);f.p||(b.id=a.sTableId+"_paginate",a.aoDrawCallback.push({fn:function(a){if(d){var b=a._iDisplayStart,i=a._iDisplayLength,h=a.fnRecordsDisplay(),l=-1===i,b=l?0:Math.ceil(b/i),i=l?1:Math.ceil(h/i),h=c(b,i),k,l=0;for(k=f.p.length;l<k;l++)Na(a,"pageButton")(a,f.p[l],l,h,b,i)}else c.fnUpdate(a,e)},sName:"pagination"}));return b}function Ta(a,b,c){var d=a._iDisplayStart,e=a._iDisplayLength,f=a.fnRecordsDisplay();0===f||-1===e?d=0:"number"===typeof b?(d=b*e,d>f&&(d=0)):"first"==b?d=0:
+"previous"==b?(d=0<=e?d-e:0,0>d&&(d=0)):"next"==b?d+e<f&&(d+=e):"last"==b?d=Math.floor((f-1)/e)*e:L(a,0,"Unknown paging action: "+b,5);b=a._iDisplayStart!==d;a._iDisplayStart=d;b&&(u(a,null,"page",[a]),c&&O(a));return b}function qb(a){return h("<div/>",{id:!a.aanFeatures.r?a.sTableId+"_processing":null,"class":a.oClasses.sProcessing}).html(a.oLanguage.sProcessing).insertBefore(a.nTable)[0]}function C(a,b){a.oFeatures.bProcessing&&h(a.aanFeatures.r).css("display",b?"block":"none");u(a,null,"processing",
+[a,b])}function rb(a){var b=h(a.nTable);b.attr("role","grid");var c=a.oScroll;if(""===c.sX&&""===c.sY)return a.nTable;var d=c.sX,e=c.sY,f=a.oClasses,g=b.children("caption"),j=g.length?g[0]._captionSide:null,i=h(b[0].cloneNode(!1)),n=h(b[0].cloneNode(!1)),l=b.children("tfoot");l.length||(l=null);i=h("<div/>",{"class":f.sScrollWrapper}).append(h("<div/>",{"class":f.sScrollHead}).css({overflow:"hidden",position:"relative",border:0,width:d?!d?null:x(d):"100%"}).append(h("<div/>",{"class":f.sScrollHeadInner}).css({"box-sizing":"content-box",
+width:c.sXInner||"100%"}).append(i.removeAttr("id").css("margin-left",0).append("top"===j?g:null).append(b.children("thead"))))).append(h("<div/>",{"class":f.sScrollBody}).css({position:"relative",overflow:"auto",width:!d?null:x(d)}).append(b));l&&i.append(h("<div/>",{"class":f.sScrollFoot}).css({overflow:"hidden",border:0,width:d?!d?null:x(d):"100%"}).append(h("<div/>",{"class":f.sScrollFootInner}).append(n.removeAttr("id").css("margin-left",0).append("bottom"===j?g:null).append(b.children("tfoot")))));
+var b=i.children(),k=b[0],f=b[1],t=l?b[2]:null;if(d)h(f).on("scroll.DT",function(){var a=this.scrollLeft;k.scrollLeft=a;l&&(t.scrollLeft=a)});h(f).css(e&&c.bCollapse?"max-height":"height",e);a.nScrollHead=k;a.nScrollBody=f;a.nScrollFoot=t;a.aoDrawCallback.push({fn:ka,sName:"scrolling"});return i[0]}function ka(a){var b=a.oScroll,c=b.sX,d=b.sXInner,e=b.sY,b=b.iBarWidth,f=h(a.nScrollHead),g=f[0].style,j=f.children("div"),i=j[0].style,n=j.children("table"),j=a.nScrollBody,l=h(j),q=j.style,t=h(a.nScrollFoot).children("div"),
+m=t.children("table"),o=h(a.nTHead),F=h(a.nTable),p=F[0],r=p.style,u=a.nTFoot?h(a.nTFoot):null,Eb=a.oBrowser,Ua=Eb.bScrollOversize,s=G(a.aoColumns,"nTh"),P,v,w,y,z=[],A=[],B=[],C=[],D,E=function(a){a=a.style;a.paddingTop="0";a.paddingBottom="0";a.borderTopWidth="0";a.borderBottomWidth="0";a.height=0};v=j.scrollHeight>j.clientHeight;if(a.scrollBarVis!==v&&a.scrollBarVis!==k)a.scrollBarVis=v,Y(a);else{a.scrollBarVis=v;F.children("thead, tfoot").remove();u&&(w=u.clone().prependTo(F),P=u.find("tr"),w=
+w.find("tr"));y=o.clone().prependTo(F);o=o.find("tr");v=y.find("tr");y.find("th, td").removeAttr("tabindex");c||(q.width="100%",f[0].style.width="100%");h.each(qa(a,y),function(b,c){D=Z(a,b);c.style.width=a.aoColumns[D].sWidth});u&&J(function(a){a.style.width=""},w);f=F.outerWidth();if(""===c){r.width="100%";if(Ua&&(F.find("tbody").height()>j.offsetHeight||"scroll"==l.css("overflow-y")))r.width=x(F.outerWidth()-b);f=F.outerWidth()}else""!==d&&(r.width=x(d),f=F.outerWidth());J(E,v);J(function(a){B.push(a.innerHTML);
+z.push(x(h(a).css("width")))},v);J(function(a,b){if(h.inArray(a,s)!==-1)a.style.width=z[b]},o);h(v).height(0);u&&(J(E,w),J(function(a){C.push(a.innerHTML);A.push(x(h(a).css("width")))},w),J(function(a,b){a.style.width=A[b]},P),h(w).height(0));J(function(a,b){a.innerHTML='<div class="dataTables_sizing" style="height:0;overflow:hidden;">'+B[b]+"</div>";a.style.width=z[b]},v);u&&J(function(a,b){a.innerHTML='<div class="dataTables_sizing" style="height:0;overflow:hidden;">'+C[b]+"</div>";a.style.width=
+A[b]},w);if(F.outerWidth()<f){P=j.scrollHeight>j.offsetHeight||"scroll"==l.css("overflow-y")?f+b:f;if(Ua&&(j.scrollHeight>j.offsetHeight||"scroll"==l.css("overflow-y")))r.width=x(P-b);(""===c||""!==d)&&L(a,1,"Possible column misalignment",6)}else P="100%";q.width=x(P);g.width=x(P);u&&(a.nScrollFoot.style.width=x(P));!e&&Ua&&(q.height=x(p.offsetHeight+b));c=F.outerWidth();n[0].style.width=x(c);i.width=x(c);d=F.height()>j.clientHeight||"scroll"==l.css("overflow-y");e="padding"+(Eb.bScrollbarLeft?"Left":
+"Right");i[e]=d?b+"px":"0px";u&&(m[0].style.width=x(c),t[0].style.width=x(c),t[0].style[e]=d?b+"px":"0px");F.children("colgroup").insertBefore(F.children("thead"));l.scroll();if((a.bSorted||a.bFiltered)&&!a._drawHold)j.scrollTop=0}}function J(a,b,c){for(var d=0,e=0,f=b.length,g,j;e<f;){g=b[e].firstChild;for(j=c?c[e].firstChild:null;g;)1===g.nodeType&&(c?a(g,j,d):a(g,d),d++),g=g.nextSibling,j=c?j.nextSibling:null;e++}}function Fa(a){var b=a.nTable,c=a.aoColumns,d=a.oScroll,e=d.sY,f=d.sX,g=d.sXInner,
+j=c.length,i=la(a,"bVisible"),n=h("th",a.nTHead),l=b.getAttribute("width"),k=b.parentNode,t=!1,m,o,p=a.oBrowser,d=p.bScrollOversize;(m=b.style.width)&&-1!==m.indexOf("%")&&(l=m);for(m=0;m<i.length;m++)o=c[i[m]],null!==o.sWidth&&(o.sWidth=Fb(o.sWidthOrig,k),t=!0);if(d||!t&&!f&&!e&&j==aa(a)&&j==n.length)for(m=0;m<j;m++)i=Z(a,m),null!==i&&(c[i].sWidth=x(n.eq(m).width()));else{j=h(b).clone().css("visibility","hidden").removeAttr("id");j.find("tbody tr").remove();var r=h("<tr/>").appendTo(j.find("tbody"));
+j.find("thead, tfoot").remove();j.append(h(a.nTHead).clone()).append(h(a.nTFoot).clone());j.find("tfoot th, tfoot td").css("width","");n=qa(a,j.find("thead")[0]);for(m=0;m<i.length;m++)o=c[i[m]],n[m].style.width=null!==o.sWidthOrig&&""!==o.sWidthOrig?x(o.sWidthOrig):"",o.sWidthOrig&&f&&h(n[m]).append(h("<div/>").css({width:o.sWidthOrig,margin:0,padding:0,border:0,height:1}));if(a.aoData.length)for(m=0;m<i.length;m++)t=i[m],o=c[t],h(Gb(a,t)).clone(!1).append(o.sContentPadding).appendTo(r);h("[name]",
+j).removeAttr("name");o=h("<div/>").css(f||e?{position:"absolute",top:0,left:0,height:1,right:0,overflow:"hidden"}:{}).append(j).appendTo(k);f&&g?j.width(g):f?(j.css("width","auto"),j.removeAttr("width"),j.width()<k.clientWidth&&l&&j.width(k.clientWidth)):e?j.width(k.clientWidth):l&&j.width(l);for(m=e=0;m<i.length;m++)k=h(n[m]),g=k.outerWidth()-k.width(),k=p.bBounding?Math.ceil(n[m].getBoundingClientRect().width):k.outerWidth(),e+=k,c[i[m]].sWidth=x(k-g);b.style.width=x(e);o.remove()}l&&(b.style.width=
+x(l));if((l||f)&&!a._reszEvt)b=function(){h(D).bind("resize.DT-"+a.sInstance,Oa(function(){Y(a)}))},d?setTimeout(b,1E3):b(),a._reszEvt=!0}function Fb(a,b){if(!a)return 0;var c=h("<div/>").css("width",x(a)).appendTo(b||I.body),d=c[0].offsetWidth;c.remove();return d}function Gb(a,b){var c=Hb(a,b);if(0>c)return null;var d=a.aoData[c];return!d.nTr?h("<td/>").html(B(a,c,b,"display"))[0]:d.anCells[b]}function Hb(a,b){for(var c,d=-1,e=-1,f=0,g=a.aoData.length;f<g;f++)c=B(a,f,b,"display")+"",c=c.replace($b,
+""),c=c.replace(/&nbsp;/g," "),c.length>d&&(d=c.length,e=f);return e}function x(a){return null===a?"0px":"number"==typeof a?0>a?"0px":a+"px":a.match(/\d$/)?a+"px":a}function V(a){var b,c,d=[],e=a.aoColumns,f,g,j,i;b=a.aaSortingFixed;c=h.isPlainObject(b);var n=[];f=function(a){a.length&&!h.isArray(a[0])?n.push(a):h.merge(n,a)};h.isArray(b)&&f(b);c&&b.pre&&f(b.pre);f(a.aaSorting);c&&b.post&&f(b.post);for(a=0;a<n.length;a++){i=n[a][0];f=e[i].aDataSort;b=0;for(c=f.length;b<c;b++)g=f[b],j=e[g].sType||
+"string",n[a]._idx===k&&(n[a]._idx=h.inArray(n[a][1],e[g].asSorting)),d.push({src:i,col:g,dir:n[a][1],index:n[a]._idx,type:j,formatter:m.ext.type.order[j+"-pre"]})}return d}function mb(a){var b,c,d=[],e=m.ext.type.order,f=a.aoData,g=0,j,i=a.aiDisplayMaster,h;Ga(a);h=V(a);b=0;for(c=h.length;b<c;b++)j=h[b],j.formatter&&g++,Ib(a,j.col);if("ssp"!=y(a)&&0!==h.length){b=0;for(c=i.length;b<c;b++)d[i[b]]=b;g===h.length?i.sort(function(a,b){var c,e,g,j,i=h.length,k=f[a]._aSortData,m=f[b]._aSortData;for(g=
+0;g<i;g++)if(j=h[g],c=k[j.col],e=m[j.col],c=c<e?-1:c>e?1:0,0!==c)return"asc"===j.dir?c:-c;c=d[a];e=d[b];return c<e?-1:c>e?1:0}):i.sort(function(a,b){var c,g,j,i,k=h.length,m=f[a]._aSortData,p=f[b]._aSortData;for(j=0;j<k;j++)if(i=h[j],c=m[i.col],g=p[i.col],i=e[i.type+"-"+i.dir]||e["string-"+i.dir],c=i(c,g),0!==c)return c;c=d[a];g=d[b];return c<g?-1:c>g?1:0})}a.bSorted=!0}function Jb(a){for(var b,c,d=a.aoColumns,e=V(a),a=a.oLanguage.oAria,f=0,g=d.length;f<g;f++){c=d[f];var j=c.asSorting;b=c.sTitle.replace(/<.*?>/g,
+"");var i=c.nTh;i.removeAttribute("aria-sort");c.bSortable&&(0<e.length&&e[0].col==f?(i.setAttribute("aria-sort","asc"==e[0].dir?"ascending":"descending"),c=j[e[0].index+1]||j[0]):c=j[0],b+="asc"===c?a.sSortAscending:a.sSortDescending);i.setAttribute("aria-label",b)}}function Va(a,b,c,d){var e=a.aaSorting,f=a.aoColumns[b].asSorting,g=function(a,b){var c=a._idx;c===k&&(c=h.inArray(a[1],f));return c+1<f.length?c+1:b?null:0};"number"===typeof e[0]&&(e=a.aaSorting=[e]);c&&a.oFeatures.bSortMulti?(c=h.inArray(b,
+G(e,"0")),-1!==c?(b=g(e[c],!0),null===b&&1===e.length&&(b=0),null===b?e.splice(c,1):(e[c][1]=f[b],e[c]._idx=b)):(e.push([b,f[0],0]),e[e.length-1]._idx=0)):e.length&&e[0][0]==b?(b=g(e[0]),e.length=1,e[0][1]=f[b],e[0]._idx=b):(e.length=0,e.push([b,f[0]]),e[0]._idx=0);T(a);"function"==typeof d&&d(a)}function Ma(a,b,c,d){var e=a.aoColumns[c];Wa(b,{},function(b){!1!==e.bSortable&&(a.oFeatures.bProcessing?(C(a,!0),setTimeout(function(){Va(a,c,b.shiftKey,d);"ssp"!==y(a)&&C(a,!1)},0)):Va(a,c,b.shiftKey,d))})}
+function va(a){var b=a.aLastSort,c=a.oClasses.sSortColumn,d=V(a),e=a.oFeatures,f,g;if(e.bSort&&e.bSortClasses){e=0;for(f=b.length;e<f;e++)g=b[e].src,h(G(a.aoData,"anCells",g)).removeClass(c+(2>e?e+1:3));e=0;for(f=d.length;e<f;e++)g=d[e].src,h(G(a.aoData,"anCells",g)).addClass(c+(2>e?e+1:3))}a.aLastSort=d}function Ib(a,b){var c=a.aoColumns[b],d=m.ext.order[c.sSortDataType],e;d&&(e=d.call(a.oInstance,a,b,$(a,b)));for(var f,g=m.ext.type.order[c.sType+"-pre"],j=0,i=a.aoData.length;j<i;j++)if(c=a.aoData[j],
+c._aSortData||(c._aSortData=[]),!c._aSortData[b]||d)f=d?e[j]:B(a,j,b,"sort"),c._aSortData[b]=g?g(f):f}function wa(a){if(a.oFeatures.bStateSave&&!a.bDestroying){var b={time:+new Date,start:a._iDisplayStart,length:a._iDisplayLength,order:h.extend(!0,[],a.aaSorting),search:Ab(a.oPreviousSearch),columns:h.map(a.aoColumns,function(b,d){return{visible:b.bVisible,search:Ab(a.aoPreSearchCols[d])}})};u(a,"aoStateSaveParams","stateSaveParams",[a,b]);a.oSavedState=b;a.fnStateSaveCallback.call(a.oInstance,a,
+b)}}function Kb(a){var b,c,d=a.aoColumns;if(a.oFeatures.bStateSave){var e=a.fnStateLoadCallback.call(a.oInstance,a);if(e&&e.time&&(b=u(a,"aoStateLoadParams","stateLoadParams",[a,e]),-1===h.inArray(!1,b)&&(b=a.iStateDuration,!(0<b&&e.time<+new Date-1E3*b)&&d.length===e.columns.length))){a.oLoadedState=h.extend(!0,{},e);e.start!==k&&(a._iDisplayStart=e.start,a.iInitDisplayStart=e.start);e.length!==k&&(a._iDisplayLength=e.length);e.order!==k&&(a.aaSorting=[],h.each(e.order,function(b,c){a.aaSorting.push(c[0]>=
+d.length?[0,c[1]]:c)}));e.search!==k&&h.extend(a.oPreviousSearch,Bb(e.search));b=0;for(c=e.columns.length;b<c;b++){var f=e.columns[b];f.visible!==k&&(d[b].bVisible=f.visible);f.search!==k&&h.extend(a.aoPreSearchCols[b],Bb(f.search))}u(a,"aoStateLoaded","stateLoaded",[a,e])}}}function xa(a){var b=m.settings,a=h.inArray(a,G(b,"nTable"));return-1!==a?b[a]:null}function L(a,b,c,d){c="DataTables warning: "+(a?"table id="+a.sTableId+" - ":"")+c;d&&(c+=". For more information about this error, please see http://datatables.net/tn/"+
+d);if(b)D.console&&console.log&&console.log(c);else if(b=m.ext,b=b.sErrMode||b.errMode,a&&u(a,null,"error",[a,d,c]),"alert"==b)alert(c);else{if("throw"==b)throw Error(c);"function"==typeof b&&b(a,d,c)}}function E(a,b,c,d){h.isArray(c)?h.each(c,function(c,d){h.isArray(d)?E(a,b,d[0],d[1]):E(a,b,d)}):(d===k&&(d=c),b[c]!==k&&(a[d]=b[c]))}function Lb(a,b,c){var d,e;for(e in b)b.hasOwnProperty(e)&&(d=b[e],h.isPlainObject(d)?(h.isPlainObject(a[e])||(a[e]={}),h.extend(!0,a[e],d)):a[e]=c&&"data"!==e&&"aaData"!==
+e&&h.isArray(d)?d.slice():d);return a}function Wa(a,b,c){h(a).bind("click.DT",b,function(b){a.blur();c(b)}).bind("keypress.DT",b,function(a){13===a.which&&(a.preventDefault(),c(a))}).bind("selectstart.DT",function(){return!1})}function z(a,b,c,d){c&&a[b].push({fn:c,sName:d})}function u(a,b,c,d){var e=[];b&&(e=h.map(a[b].slice().reverse(),function(b){return b.fn.apply(a.oInstance,d)}));null!==c&&(b=h.Event(c+".dt"),h(a.nTable).trigger(b,d),e.push(b.result));return e}function Sa(a){var b=a._iDisplayStart,
+c=a.fnDisplayEnd(),d=a._iDisplayLength;b>=c&&(b=c-d);b-=b%d;if(-1===d||0>b)b=0;a._iDisplayStart=b}function Na(a,b){var c=a.renderer,d=m.ext.renderer[b];return h.isPlainObject(c)&&c[b]?d[c[b]]||d._:"string"===typeof c?d[c]||d._:d._}function y(a){return a.oFeatures.bServerSide?"ssp":a.ajax||a.sAjaxSource?"ajax":"dom"}function ya(a,b){var c=[],c=Mb.numbers_length,d=Math.floor(c/2);b<=c?c=W(0,b):a<=d?(c=W(0,c-2),c.push("ellipsis"),c.push(b-1)):(a>=b-1-d?c=W(b-(c-2),b):(c=W(a-d+2,a+d-1),c.push("ellipsis"),
+c.push(b-1)),c.splice(0,0,"ellipsis"),c.splice(0,0,0));c.DT_el="span";return c}function db(a){h.each({num:function(b){return za(b,a)},"num-fmt":function(b){return za(b,a,Xa)},"html-num":function(b){return za(b,a,Aa)},"html-num-fmt":function(b){return za(b,a,Aa,Xa)}},function(b,c){v.type.order[b+a+"-pre"]=c;b.match(/^html\-/)&&(v.type.search[b+a]=v.type.search.html)})}function Nb(a){return function(){var b=[xa(this[m.ext.iApiIndex])].concat(Array.prototype.slice.call(arguments));return m.ext.internal[a].apply(this,
+b)}}var m=function(a){this.$=function(a,b){return this.api(!0).$(a,b)};this._=function(a,b){return this.api(!0).rows(a,b).data()};this.api=function(a){return a?new r(xa(this[v.iApiIndex])):new r(this)};this.fnAddData=function(a,b){var c=this.api(!0),d=h.isArray(a)&&(h.isArray(a[0])||h.isPlainObject(a[0]))?c.rows.add(a):c.row.add(a);(b===k||b)&&c.draw();return d.flatten().toArray()};this.fnAdjustColumnSizing=function(a){var b=this.api(!0).columns.adjust(),c=b.settings()[0],d=c.oScroll;a===k||a?b.draw(!1):
+(""!==d.sX||""!==d.sY)&&ka(c)};this.fnClearTable=function(a){var b=this.api(!0).clear();(a===k||a)&&b.draw()};this.fnClose=function(a){this.api(!0).row(a).child.hide()};this.fnDeleteRow=function(a,b,c){var d=this.api(!0),a=d.rows(a),e=a.settings()[0],h=e.aoData[a[0][0]];a.remove();b&&b.call(this,e,h);(c===k||c)&&d.draw();return h};this.fnDestroy=function(a){this.api(!0).destroy(a)};this.fnDraw=function(a){this.api(!0).draw(a)};this.fnFilter=function(a,b,c,d,e,h){e=this.api(!0);null===b||b===k?e.search(a,
+c,d,h):e.column(b).search(a,c,d,h);e.draw()};this.fnGetData=function(a,b){var c=this.api(!0);if(a!==k){var d=a.nodeName?a.nodeName.toLowerCase():"";return b!==k||"td"==d||"th"==d?c.cell(a,b).data():c.row(a).data()||null}return c.data().toArray()};this.fnGetNodes=function(a){var b=this.api(!0);return a!==k?b.row(a).node():b.rows().nodes().flatten().toArray()};this.fnGetPosition=function(a){var b=this.api(!0),c=a.nodeName.toUpperCase();return"TR"==c?b.row(a).index():"TD"==c||"TH"==c?(a=b.cell(a).index(),
+[a.row,a.columnVisible,a.column]):null};this.fnIsOpen=function(a){return this.api(!0).row(a).child.isShown()};this.fnOpen=function(a,b,c){return this.api(!0).row(a).child(b,c).show().child()[0]};this.fnPageChange=function(a,b){var c=this.api(!0).page(a);(b===k||b)&&c.draw(!1)};this.fnSetColumnVis=function(a,b,c){a=this.api(!0).column(a).visible(b);(c===k||c)&&a.columns.adjust().draw()};this.fnSettings=function(){return xa(this[v.iApiIndex])};this.fnSort=function(a){this.api(!0).order(a).draw()};this.fnSortListener=
+function(a,b,c){this.api(!0).order.listener(a,b,c)};this.fnUpdate=function(a,b,c,d,e){var h=this.api(!0);c===k||null===c?h.row(b).data(a):h.cell(b,c).data(a);(e===k||e)&&h.columns.adjust();(d===k||d)&&h.draw();return 0};this.fnVersionCheck=v.fnVersionCheck;var b=this,c=a===k,d=this.length;c&&(a={});this.oApi=this.internal=v.internal;for(var e in m.ext.internal)e&&(this[e]=Nb(e));this.each(function(){var e={},e=1<d?Lb(e,a,!0):a,g=0,j,i=this.getAttribute("id"),n=!1,l=m.defaults,q=h(this);if("table"!=
+this.nodeName.toLowerCase())L(null,0,"Non-table node initialisation ("+this.nodeName+")",2);else{eb(l);fb(l.column);K(l,l,!0);K(l.column,l.column,!0);K(l,h.extend(e,q.data()));var t=m.settings,g=0;for(j=t.length;g<j;g++){var p=t[g];if(p.nTable==this||p.nTHead.parentNode==this||p.nTFoot&&p.nTFoot.parentNode==this){g=e.bRetrieve!==k?e.bRetrieve:l.bRetrieve;if(c||g)return p.oInstance;if(e.bDestroy!==k?e.bDestroy:l.bDestroy){p.oInstance.fnDestroy();break}else{L(p,0,"Cannot reinitialise DataTable",3);
+return}}if(p.sTableId==this.id){t.splice(g,1);break}}if(null===i||""===i)this.id=i="DataTables_Table_"+m.ext._unique++;var o=h.extend(!0,{},m.models.oSettings,{sDestroyWidth:q[0].style.width,sInstance:i,sTableId:i});o.nTable=this;o.oApi=b.internal;o.oInit=e;t.push(o);o.oInstance=1===b.length?b:q.dataTable();eb(e);e.oLanguage&&Da(e.oLanguage);e.aLengthMenu&&!e.iDisplayLength&&(e.iDisplayLength=h.isArray(e.aLengthMenu[0])?e.aLengthMenu[0][0]:e.aLengthMenu[0]);e=Lb(h.extend(!0,{},l),e);E(o.oFeatures,
+e,"bPaginate bLengthChange bFilter bSort bSortMulti bInfo bProcessing bAutoWidth bSortClasses bServerSide bDeferRender".split(" "));E(o,e,["asStripeClasses","ajax","fnServerData","fnFormatNumber","sServerMethod","aaSorting","aaSortingFixed","aLengthMenu","sPaginationType","sAjaxSource","sAjaxDataProp","iStateDuration","sDom","bSortCellsTop","iTabIndex","fnStateLoadCallback","fnStateSaveCallback","renderer","searchDelay","rowId",["iCookieDuration","iStateDuration"],["oSearch","oPreviousSearch"],["aoSearchCols",
+"aoPreSearchCols"],["iDisplayLength","_iDisplayLength"],["bJQueryUI","bJUI"]]);E(o.oScroll,e,[["sScrollX","sX"],["sScrollXInner","sXInner"],["sScrollY","sY"],["bScrollCollapse","bCollapse"]]);E(o.oLanguage,e,"fnInfoCallback");z(o,"aoDrawCallback",e.fnDrawCallback,"user");z(o,"aoServerParams",e.fnServerParams,"user");z(o,"aoStateSaveParams",e.fnStateSaveParams,"user");z(o,"aoStateLoadParams",e.fnStateLoadParams,"user");z(o,"aoStateLoaded",e.fnStateLoaded,"user");z(o,"aoRowCallback",e.fnRowCallback,
+"user");z(o,"aoRowCreatedCallback",e.fnCreatedRow,"user");z(o,"aoHeaderCallback",e.fnHeaderCallback,"user");z(o,"aoFooterCallback",e.fnFooterCallback,"user");z(o,"aoInitComplete",e.fnInitComplete,"user");z(o,"aoPreDrawCallback",e.fnPreDrawCallback,"user");o.rowIdFn=Q(e.rowId);gb(o);i=o.oClasses;e.bJQueryUI?(h.extend(i,m.ext.oJUIClasses,e.oClasses),e.sDom===l.sDom&&"lfrtip"===l.sDom&&(o.sDom='<"H"lfr>t<"F"ip>'),o.renderer)?h.isPlainObject(o.renderer)&&!o.renderer.header&&(o.renderer.header="jqueryui"):
+o.renderer="jqueryui":h.extend(i,m.ext.classes,e.oClasses);q.addClass(i.sTable);o.iInitDisplayStart===k&&(o.iInitDisplayStart=e.iDisplayStart,o._iDisplayStart=e.iDisplayStart);null!==e.iDeferLoading&&(o.bDeferLoading=!0,g=h.isArray(e.iDeferLoading),o._iRecordsDisplay=g?e.iDeferLoading[0]:e.iDeferLoading,o._iRecordsTotal=g?e.iDeferLoading[1]:e.iDeferLoading);var r=o.oLanguage;h.extend(!0,r,e.oLanguage);""!==r.sUrl&&(h.ajax({dataType:"json",url:r.sUrl,success:function(a){Da(a);K(l.oLanguage,a);h.extend(true,
+r,a);ga(o)},error:function(){ga(o)}}),n=!0);null===e.asStripeClasses&&(o.asStripeClasses=[i.sStripeOdd,i.sStripeEven]);var g=o.asStripeClasses,v=q.children("tbody").find("tr").eq(0);-1!==h.inArray(!0,h.map(g,function(a){return v.hasClass(a)}))&&(h("tbody tr",this).removeClass(g.join(" ")),o.asDestroyStripes=g.slice());t=[];g=this.getElementsByTagName("thead");0!==g.length&&(da(o.aoHeader,g[0]),t=qa(o));if(null===e.aoColumns){p=[];g=0;for(j=t.length;g<j;g++)p.push(null)}else p=e.aoColumns;g=0;for(j=
+p.length;g<j;g++)Ea(o,t?t[g]:null);ib(o,e.aoColumnDefs,p,function(a,b){ja(o,a,b)});if(v.length){var s=function(a,b){return a.getAttribute("data-"+b)!==null?b:null};h(v[0]).children("th, td").each(function(a,b){var c=o.aoColumns[a];if(c.mData===a){var d=s(b,"sort")||s(b,"order"),e=s(b,"filter")||s(b,"search");if(d!==null||e!==null){c.mData={_:a+".display",sort:d!==null?a+".@data-"+d:k,type:d!==null?a+".@data-"+d:k,filter:e!==null?a+".@data-"+e:k};ja(o,a)}}})}var w=o.oFeatures;e.bStateSave&&(w.bStateSave=
+!0,Kb(o,e),z(o,"aoDrawCallback",wa,"state_save"));if(e.aaSorting===k){t=o.aaSorting;g=0;for(j=t.length;g<j;g++)t[g][1]=o.aoColumns[g].asSorting[0]}va(o);w.bSort&&z(o,"aoDrawCallback",function(){if(o.bSorted){var a=V(o),b={};h.each(a,function(a,c){b[c.src]=c.dir});u(o,null,"order",[o,a,b]);Jb(o)}});z(o,"aoDrawCallback",function(){(o.bSorted||y(o)==="ssp"||w.bDeferRender)&&va(o)},"sc");g=q.children("caption").each(function(){this._captionSide=q.css("caption-side")});j=q.children("thead");0===j.length&&
+(j=h("<thead/>").appendTo(this));o.nTHead=j[0];j=q.children("tbody");0===j.length&&(j=h("<tbody/>").appendTo(this));o.nTBody=j[0];j=q.children("tfoot");if(0===j.length&&0<g.length&&(""!==o.oScroll.sX||""!==o.oScroll.sY))j=h("<tfoot/>").appendTo(this);0===j.length||0===j.children().length?q.addClass(i.sNoFooter):0<j.length&&(o.nTFoot=j[0],da(o.aoFooter,o.nTFoot));if(e.aaData)for(g=0;g<e.aaData.length;g++)N(o,e.aaData[g]);else(o.bDeferLoading||"dom"==y(o))&&ma(o,h(o.nTBody).children("tr"));o.aiDisplay=
+o.aiDisplayMaster.slice();o.bInitialised=!0;!1===n&&ga(o)}});b=null;return this},v,r,p,s,Ya={},Ob=/[\r\n]/g,Aa=/<.*?>/g,ac=/^[\w\+\-]/,bc=/[\w\+\-]$/,cc=RegExp("(\\/|\\.|\\*|\\+|\\?|\\||\\(|\\)|\\[|\\]|\\{|\\}|\\\\|\\$|\\^|\\-)","g"),Xa=/[',$£€¥%\u2009\u202F\u20BD\u20a9\u20BArfk]/gi,M=function(a){return!a||!0===a||"-"===a?!0:!1},Pb=function(a){var b=parseInt(a,10);return!isNaN(b)&&isFinite(a)?b:null},Qb=function(a,b){Ya[b]||(Ya[b]=RegExp(Qa(b),"g"));return"string"===typeof a&&"."!==b?a.replace(/\./g,
+"").replace(Ya[b],"."):a},Za=function(a,b,c){var d="string"===typeof a;if(M(a))return!0;b&&d&&(a=Qb(a,b));c&&d&&(a=a.replace(Xa,""));return!isNaN(parseFloat(a))&&isFinite(a)},Rb=function(a,b,c){return M(a)?!0:!(M(a)||"string"===typeof a)?null:Za(a.replace(Aa,""),b,c)?!0:null},G=function(a,b,c){var d=[],e=0,f=a.length;if(c!==k)for(;e<f;e++)a[e]&&a[e][b]&&d.push(a[e][b][c]);else for(;e<f;e++)a[e]&&d.push(a[e][b]);return d},ha=function(a,b,c,d){var e=[],f=0,g=b.length;if(d!==k)for(;f<g;f++)a[b[f]][c]&&
+e.push(a[b[f]][c][d]);else for(;f<g;f++)e.push(a[b[f]][c]);return e},W=function(a,b){var c=[],d;b===k?(b=0,d=a):(d=b,b=a);for(var e=b;e<d;e++)c.push(e);return c},Sb=function(a){for(var b=[],c=0,d=a.length;c<d;c++)a[c]&&b.push(a[c]);return b},pa=function(a){var b=[],c,d,e=a.length,f,g=0;d=0;a:for(;d<e;d++){c=a[d];for(f=0;f<g;f++)if(b[f]===c)continue a;b.push(c);g++}return b};m.util={throttle:function(a,b){var c=b!==k?b:200,d,e;return function(){var b=this,g=+new Date,h=arguments;d&&g<d+c?(clearTimeout(e),
+e=setTimeout(function(){d=k;a.apply(b,h)},c)):(d=g,a.apply(b,h))}},escapeRegex:function(a){return a.replace(cc,"\\$1")}};var A=function(a,b,c){a[b]!==k&&(a[c]=a[b])},ba=/\[.*?\]$/,U=/\(\)$/,Qa=m.util.escapeRegex,ua=h("<div>")[0],Zb=ua.textContent!==k,$b=/<.*?>/g,Oa=m.util.throttle,Tb=[],w=Array.prototype,dc=function(a){var b,c,d=m.settings,e=h.map(d,function(a){return a.nTable});if(a){if(a.nTable&&a.oApi)return[a];if(a.nodeName&&"table"===a.nodeName.toLowerCase())return b=h.inArray(a,e),-1!==b?[d[b]]:
+null;if(a&&"function"===typeof a.settings)return a.settings().toArray();"string"===typeof a?c=h(a):a instanceof h&&(c=a)}else return[];if(c)return c.map(function(){b=h.inArray(this,e);return-1!==b?d[b]:null}).toArray()};r=function(a,b){if(!(this instanceof r))return new r(a,b);var c=[],d=function(a){(a=dc(a))&&(c=c.concat(a))};if(h.isArray(a))for(var e=0,f=a.length;e<f;e++)d(a[e]);else d(a);this.context=pa(c);b&&h.merge(this,b);this.selector={rows:null,cols:null,opts:null};r.extend(this,this,Tb)};
+m.Api=r;h.extend(r.prototype,{any:function(){return 0!==this.count()},concat:w.concat,context:[],count:function(){return this.flatten().length},each:function(a){for(var b=0,c=this.length;b<c;b++)a.call(this,this[b],b,this);return this},eq:function(a){var b=this.context;return b.length>a?new r(b[a],this[a]):null},filter:function(a){var b=[];if(w.filter)b=w.filter.call(this,a,this);else for(var c=0,d=this.length;c<d;c++)a.call(this,this[c],c,this)&&b.push(this[c]);return new r(this.context,b)},flatten:function(){var a=
+[];return new r(this.context,a.concat.apply(a,this.toArray()))},join:w.join,indexOf:w.indexOf||function(a,b){for(var c=b||0,d=this.length;c<d;c++)if(this[c]===a)return c;return-1},iterator:function(a,b,c,d){var e=[],f,g,h,i,n,l=this.context,m,t,p=this.selector;"string"===typeof a&&(d=c,c=b,b=a,a=!1);g=0;for(h=l.length;g<h;g++){var o=new r(l[g]);if("table"===b)f=c.call(o,l[g],g),f!==k&&e.push(f);else if("columns"===b||"rows"===b)f=c.call(o,l[g],this[g],g),f!==k&&e.push(f);else if("column"===b||"column-rows"===
+b||"row"===b||"cell"===b){t=this[g];"column-rows"===b&&(m=Ba(l[g],p.opts));i=0;for(n=t.length;i<n;i++)f=t[i],f="cell"===b?c.call(o,l[g],f.row,f.column,g,i):c.call(o,l[g],f,g,i,m),f!==k&&e.push(f)}}return e.length||d?(a=new r(l,a?e.concat.apply([],e):e),b=a.selector,b.rows=p.rows,b.cols=p.cols,b.opts=p.opts,a):this},lastIndexOf:w.lastIndexOf||function(a,b){return this.indexOf.apply(this.toArray.reverse(),arguments)},length:0,map:function(a){var b=[];if(w.map)b=w.map.call(this,a,this);else for(var c=
+0,d=this.length;c<d;c++)b.push(a.call(this,this[c],c));return new r(this.context,b)},pluck:function(a){return this.map(function(b){return b[a]})},pop:w.pop,push:w.push,reduce:w.reduce||function(a,b){return hb(this,a,b,0,this.length,1)},reduceRight:w.reduceRight||function(a,b){return hb(this,a,b,this.length-1,-1,-1)},reverse:w.reverse,selector:null,shift:w.shift,sort:w.sort,splice:w.splice,toArray:function(){return w.slice.call(this)},to$:function(){return h(this)},toJQuery:function(){return h(this)},
+unique:function(){return new r(this.context,pa(this))},unshift:w.unshift});r.extend=function(a,b,c){if(c.length&&b&&(b instanceof r||b.__dt_wrapper)){var d,e,f,g=function(a,b,c){return function(){var d=b.apply(a,arguments);r.extend(d,d,c.methodExt);return d}};d=0;for(e=c.length;d<e;d++)f=c[d],b[f.name]="function"===typeof f.val?g(a,f.val,f):h.isPlainObject(f.val)?{}:f.val,b[f.name].__dt_wrapper=!0,r.extend(a,b[f.name],f.propExt)}};r.register=p=function(a,b){if(h.isArray(a))for(var c=0,d=a.length;c<
+d;c++)r.register(a[c],b);else for(var e=a.split("."),f=Tb,g,j,c=0,d=e.length;c<d;c++){g=(j=-1!==e[c].indexOf("()"))?e[c].replace("()",""):e[c];var i;a:{i=0;for(var n=f.length;i<n;i++)if(f[i].name===g){i=f[i];break a}i=null}i||(i={name:g,val:{},methodExt:[],propExt:[]},f.push(i));c===d-1?i.val=b:f=j?i.methodExt:i.propExt}};r.registerPlural=s=function(a,b,c){r.register(a,c);r.register(b,function(){var a=c.apply(this,arguments);return a===this?this:a instanceof r?a.length?h.isArray(a[0])?new r(a.context,
+a[0]):a[0]:k:a})};p("tables()",function(a){var b;if(a){b=r;var c=this.context;if("number"===typeof a)a=[c[a]];else var d=h.map(c,function(a){return a.nTable}),a=h(d).filter(a).map(function(){var a=h.inArray(this,d);return c[a]}).toArray();b=new b(a)}else b=this;return b});p("table()",function(a){var a=this.tables(a),b=a.context;return b.length?new r(b[0]):a});s("tables().nodes()","table().node()",function(){return this.iterator("table",function(a){return a.nTable},1)});s("tables().body()","table().body()",
+function(){return this.iterator("table",function(a){return a.nTBody},1)});s("tables().header()","table().header()",function(){return this.iterator("table",function(a){return a.nTHead},1)});s("tables().footer()","table().footer()",function(){return this.iterator("table",function(a){return a.nTFoot},1)});s("tables().containers()","table().container()",function(){return this.iterator("table",function(a){return a.nTableWrapper},1)});p("draw()",function(a){return this.iterator("table",function(b){"page"===
+a?O(b):("string"===typeof a&&(a="full-hold"===a?!1:!0),T(b,!1===a))})});p("page()",function(a){return a===k?this.page.info().page:this.iterator("table",function(b){Ta(b,a)})});p("page.info()",function(){if(0===this.context.length)return k;var a=this.context[0],b=a._iDisplayStart,c=a.oFeatures.bPaginate?a._iDisplayLength:-1,d=a.fnRecordsDisplay(),e=-1===c;return{page:e?0:Math.floor(b/c),pages:e?1:Math.ceil(d/c),start:b,end:a.fnDisplayEnd(),length:c,recordsTotal:a.fnRecordsTotal(),recordsDisplay:d,
+serverSide:"ssp"===y(a)}});p("page.len()",function(a){return a===k?0!==this.context.length?this.context[0]._iDisplayLength:k:this.iterator("table",function(b){Ra(b,a)})});var Ub=function(a,b,c){if(c){var d=new r(a);d.one("draw",function(){c(d.ajax.json())})}if("ssp"==y(a))T(a,b);else{C(a,!0);var e=a.jqXHR;e&&4!==e.readyState&&e.abort();ra(a,[],function(c){na(a);for(var c=sa(a,c),d=0,e=c.length;d<e;d++)N(a,c[d]);T(a,b);C(a,!1)})}};p("ajax.json()",function(){var a=this.context;if(0<a.length)return a[0].json});
+p("ajax.params()",function(){var a=this.context;if(0<a.length)return a[0].oAjaxData});p("ajax.reload()",function(a,b){return this.iterator("table",function(c){Ub(c,!1===b,a)})});p("ajax.url()",function(a){var b=this.context;if(a===k){if(0===b.length)return k;b=b[0];return b.ajax?h.isPlainObject(b.ajax)?b.ajax.url:b.ajax:b.sAjaxSource}return this.iterator("table",function(b){h.isPlainObject(b.ajax)?b.ajax.url=a:b.ajax=a})});p("ajax.url().load()",function(a,b){return this.iterator("table",function(c){Ub(c,
+!1===b,a)})});var $a=function(a,b,c,d,e){var f=[],g,j,i,n,l,m;i=typeof b;if(!b||"string"===i||"function"===i||b.length===k)b=[b];i=0;for(n=b.length;i<n;i++){j=b[i]&&b[i].split?b[i].split(","):[b[i]];l=0;for(m=j.length;l<m;l++)(g=c("string"===typeof j[l]?h.trim(j[l]):j[l]))&&g.length&&(f=f.concat(g))}a=v.selector[a];if(a.length){i=0;for(n=a.length;i<n;i++)f=a[i](d,e,f)}return pa(f)},ab=function(a){a||(a={});a.filter&&a.search===k&&(a.search=a.filter);return h.extend({search:"none",order:"current",
+page:"all"},a)},bb=function(a){for(var b=0,c=a.length;b<c;b++)if(0<a[b].length)return a[0]=a[b],a[0].length=1,a.length=1,a.context=[a.context[b]],a;a.length=0;return a},Ba=function(a,b){var c,d,e,f=[],g=a.aiDisplay;c=a.aiDisplayMaster;var j=b.search;d=b.order;e=b.page;if("ssp"==y(a))return"removed"===j?[]:W(0,c.length);if("current"==e){c=a._iDisplayStart;for(d=a.fnDisplayEnd();c<d;c++)f.push(g[c])}else if("current"==d||"applied"==d)f="none"==j?c.slice():"applied"==j?g.slice():h.map(c,function(a){return-1===
+h.inArray(a,g)?a:null});else if("index"==d||"original"==d){c=0;for(d=a.aoData.length;c<d;c++)"none"==j?f.push(c):(e=h.inArray(c,g),(-1===e&&"removed"==j||0<=e&&"applied"==j)&&f.push(c))}return f};p("rows()",function(a,b){a===k?a="":h.isPlainObject(a)&&(b=a,a="");var b=ab(b),c=this.iterator("table",function(c){var e=b;return $a("row",a,function(a){var b=Pb(a);if(b!==null&&!e)return[b];var j=Ba(c,e);if(b!==null&&h.inArray(b,j)!==-1)return[b];if(!a)return j;if(typeof a==="function")return h.map(j,function(b){var e=
+c.aoData[b];return a(b,e._aData,e.nTr)?b:null});b=Sb(ha(c.aoData,j,"nTr"));if(a.nodeName){if(a._DT_RowIndex!==k)return[a._DT_RowIndex];if(a._DT_CellIndex)return[a._DT_CellIndex.row];b=h(a).closest("*[data-dt-row]");return b.length?[b.data("dt-row")]:[]}if(typeof a==="string"&&a.charAt(0)==="#"){j=c.aIds[a.replace(/^#/,"")];if(j!==k)return[j.idx]}return h(b).filter(a).map(function(){return this._DT_RowIndex}).toArray()},c,e)},1);c.selector.rows=a;c.selector.opts=b;return c});p("rows().nodes()",function(){return this.iterator("row",
+function(a,b){return a.aoData[b].nTr||k},1)});p("rows().data()",function(){return this.iterator(!0,"rows",function(a,b){return ha(a.aoData,b,"_aData")},1)});s("rows().cache()","row().cache()",function(a){return this.iterator("row",function(b,c){var d=b.aoData[c];return"search"===a?d._aFilterData:d._aSortData},1)});s("rows().invalidate()","row().invalidate()",function(a){return this.iterator("row",function(b,c){ca(b,c,a)})});s("rows().indexes()","row().index()",function(){return this.iterator("row",
+function(a,b){return b},1)});s("rows().ids()","row().id()",function(a){for(var b=[],c=this.context,d=0,e=c.length;d<e;d++)for(var f=0,g=this[d].length;f<g;f++){var h=c[d].rowIdFn(c[d].aoData[this[d][f]]._aData);b.push((!0===a?"#":"")+h)}return new r(c,b)});s("rows().remove()","row().remove()",function(){var a=this;this.iterator("row",function(b,c,d){var e=b.aoData,f=e[c],g,h,i,n,l;e.splice(c,1);g=0;for(h=e.length;g<h;g++)if(i=e[g],l=i.anCells,null!==i.nTr&&(i.nTr._DT_RowIndex=g),null!==l){i=0;for(n=
+l.length;i<n;i++)l[i]._DT_CellIndex.row=g}oa(b.aiDisplayMaster,c);oa(b.aiDisplay,c);oa(a[d],c,!1);Sa(b);c=b.rowIdFn(f._aData);c!==k&&delete b.aIds[c]});this.iterator("table",function(a){for(var c=0,d=a.aoData.length;c<d;c++)a.aoData[c].idx=c});return this});p("rows.add()",function(a){var b=this.iterator("table",function(b){var c,f,g,h=[];f=0;for(g=a.length;f<g;f++)c=a[f],c.nodeName&&"TR"===c.nodeName.toUpperCase()?h.push(ma(b,c)[0]):h.push(N(b,c));return h},1),c=this.rows(-1);c.pop();h.merge(c,b);
+return c});p("row()",function(a,b){return bb(this.rows(a,b))});p("row().data()",function(a){var b=this.context;if(a===k)return b.length&&this.length?b[0].aoData[this[0]]._aData:k;b[0].aoData[this[0]]._aData=a;ca(b[0],this[0],"data");return this});p("row().node()",function(){var a=this.context;return a.length&&this.length?a[0].aoData[this[0]].nTr||null:null});p("row.add()",function(a){a instanceof h&&a.length&&(a=a[0]);var b=this.iterator("table",function(b){return a.nodeName&&"TR"===a.nodeName.toUpperCase()?
+ma(b,a)[0]:N(b,a)});return this.row(b[0])});var cb=function(a,b){var c=a.context;if(c.length&&(c=c[0].aoData[b!==k?b:a[0]])&&c._details)c._details.remove(),c._detailsShow=k,c._details=k},Vb=function(a,b){var c=a.context;if(c.length&&a.length){var d=c[0].aoData[a[0]];if(d._details){(d._detailsShow=b)?d._details.insertAfter(d.nTr):d._details.detach();var e=c[0],f=new r(e),g=e.aoData;f.off("draw.dt.DT_details column-visibility.dt.DT_details destroy.dt.DT_details");0<G(g,"_details").length&&(f.on("draw.dt.DT_details",
+function(a,b){e===b&&f.rows({page:"current"}).eq(0).each(function(a){a=g[a];a._detailsShow&&a._details.insertAfter(a.nTr)})}),f.on("column-visibility.dt.DT_details",function(a,b){if(e===b)for(var c,d=aa(b),f=0,h=g.length;f<h;f++)c=g[f],c._details&&c._details.children("td[colspan]").attr("colspan",d)}),f.on("destroy.dt.DT_details",function(a,b){if(e===b)for(var c=0,d=g.length;c<d;c++)g[c]._details&&cb(f,c)}))}}};p("row().child()",function(a,b){var c=this.context;if(a===k)return c.length&&this.length?
+c[0].aoData[this[0]]._details:k;if(!0===a)this.child.show();else if(!1===a)cb(this);else if(c.length&&this.length){var d=c[0],c=c[0].aoData[this[0]],e=[],f=function(a,b){if(h.isArray(a)||a instanceof h)for(var c=0,k=a.length;c<k;c++)f(a[c],b);else a.nodeName&&"tr"===a.nodeName.toLowerCase()?e.push(a):(c=h("<tr><td/></tr>").addClass(b),h("td",c).addClass(b).html(a)[0].colSpan=aa(d),e.push(c[0]))};f(a,b);c._details&&c._details.remove();c._details=h(e);c._detailsShow&&c._details.insertAfter(c.nTr)}return this});
+p(["row().child.show()","row().child().show()"],function(){Vb(this,!0);return this});p(["row().child.hide()","row().child().hide()"],function(){Vb(this,!1);return this});p(["row().child.remove()","row().child().remove()"],function(){cb(this);return this});p("row().child.isShown()",function(){var a=this.context;return a.length&&this.length?a[0].aoData[this[0]]._detailsShow||!1:!1});var ec=/^(.+):(name|visIdx|visible)$/,Wb=function(a,b,c,d,e){for(var c=[],d=0,f=e.length;d<f;d++)c.push(B(a,e[d],b));
+return c};p("columns()",function(a,b){a===k?a="":h.isPlainObject(a)&&(b=a,a="");var b=ab(b),c=this.iterator("table",function(c){var e=a,f=b,g=c.aoColumns,j=G(g,"sName"),i=G(g,"nTh");return $a("column",e,function(a){var b=Pb(a);if(a==="")return W(g.length);if(b!==null)return[b>=0?b:g.length+b];if(typeof a==="function"){var e=Ba(c,f);return h.map(g,function(b,f){return a(f,Wb(c,f,0,0,e),i[f])?f:null})}var k=typeof a==="string"?a.match(ec):"";if(k)switch(k[2]){case "visIdx":case "visible":b=parseInt(k[1],
+10);if(b<0){var m=h.map(g,function(a,b){return a.bVisible?b:null});return[m[m.length+b]]}return[Z(c,b)];case "name":return h.map(j,function(a,b){return a===k[1]?b:null});default:return[]}if(a.nodeName&&a._DT_CellIndex)return[a._DT_CellIndex.column];b=h(i).filter(a).map(function(){return h.inArray(this,i)}).toArray();if(b.length||!a.nodeName)return b;b=h(a).closest("*[data-dt-column]");return b.length?[b.data("dt-column")]:[]},c,f)},1);c.selector.cols=a;c.selector.opts=b;return c});s("columns().header()",
+"column().header()",function(){return this.iterator("column",function(a,b){return a.aoColumns[b].nTh},1)});s("columns().footer()","column().footer()",function(){return this.iterator("column",function(a,b){return a.aoColumns[b].nTf},1)});s("columns().data()","column().data()",function(){return this.iterator("column-rows",Wb,1)});s("columns().dataSrc()","column().dataSrc()",function(){return this.iterator("column",function(a,b){return a.aoColumns[b].mData},1)});s("columns().cache()","column().cache()",
+function(a){return this.iterator("column-rows",function(b,c,d,e,f){return ha(b.aoData,f,"search"===a?"_aFilterData":"_aSortData",c)},1)});s("columns().nodes()","column().nodes()",function(){return this.iterator("column-rows",function(a,b,c,d,e){return ha(a.aoData,e,"anCells",b)},1)});s("columns().visible()","column().visible()",function(a,b){var c=this.iterator("column",function(b,c){if(a===k)return b.aoColumns[c].bVisible;var f=b.aoColumns,g=f[c],j=b.aoData,i,n,l;if(a!==k&&g.bVisible!==a){if(a){var m=
+h.inArray(!0,G(f,"bVisible"),c+1);i=0;for(n=j.length;i<n;i++)l=j[i].nTr,f=j[i].anCells,l&&l.insertBefore(f[c],f[m]||null)}else h(G(b.aoData,"anCells",c)).detach();g.bVisible=a;ea(b,b.aoHeader);ea(b,b.aoFooter);wa(b)}});a!==k&&(this.iterator("column",function(c,e){u(c,null,"column-visibility",[c,e,a,b])}),(b===k||b)&&this.columns.adjust());return c});s("columns().indexes()","column().index()",function(a){return this.iterator("column",function(b,c){return"visible"===a?$(b,c):c},1)});p("columns.adjust()",
+function(){return this.iterator("table",function(a){Y(a)},1)});p("column.index()",function(a,b){if(0!==this.context.length){var c=this.context[0];if("fromVisible"===a||"toData"===a)return Z(c,b);if("fromData"===a||"toVisible"===a)return $(c,b)}});p("column()",function(a,b){return bb(this.columns(a,b))});p("cells()",function(a,b,c){h.isPlainObject(a)&&(a.row===k?(c=a,a=null):(c=b,b=null));h.isPlainObject(b)&&(c=b,b=null);if(null===b||b===k)return this.iterator("table",function(b){var d=a,e=ab(c),f=
+b.aoData,g=Ba(b,e),j=Sb(ha(f,g,"anCells")),i=h([].concat.apply([],j)),l,n=b.aoColumns.length,m,p,r,u,v,s;return $a("cell",d,function(a){var c=typeof a==="function";if(a===null||a===k||c){m=[];p=0;for(r=g.length;p<r;p++){l=g[p];for(u=0;u<n;u++){v={row:l,column:u};if(c){s=f[l];a(v,B(b,l,u),s.anCells?s.anCells[u]:null)&&m.push(v)}else m.push(v)}}return m}if(h.isPlainObject(a))return[a];c=i.filter(a).map(function(a,b){return{row:b._DT_CellIndex.row,column:b._DT_CellIndex.column}}).toArray();if(c.length||
+!a.nodeName)return c;s=h(a).closest("*[data-dt-row]");return s.length?[{row:s.data("dt-row"),column:s.data("dt-column")}]:[]},b,e)});var d=this.columns(b,c),e=this.rows(a,c),f,g,j,i,n,l=this.iterator("table",function(a,b){f=[];g=0;for(j=e[b].length;g<j;g++){i=0;for(n=d[b].length;i<n;i++)f.push({row:e[b][g],column:d[b][i]})}return f},1);h.extend(l.selector,{cols:b,rows:a,opts:c});return l});s("cells().nodes()","cell().node()",function(){return this.iterator("cell",function(a,b,c){return(a=a.aoData[b])&&
+a.anCells?a.anCells[c]:k},1)});p("cells().data()",function(){return this.iterator("cell",function(a,b,c){return B(a,b,c)},1)});s("cells().cache()","cell().cache()",function(a){a="search"===a?"_aFilterData":"_aSortData";return this.iterator("cell",function(b,c,d){return b.aoData[c][a][d]},1)});s("cells().render()","cell().render()",function(a){return this.iterator("cell",function(b,c,d){return B(b,c,d,a)},1)});s("cells().indexes()","cell().index()",function(){return this.iterator("cell",function(a,
+b,c){return{row:b,column:c,columnVisible:$(a,c)}},1)});s("cells().invalidate()","cell().invalidate()",function(a){return this.iterator("cell",function(b,c,d){ca(b,c,a,d)})});p("cell()",function(a,b,c){return bb(this.cells(a,b,c))});p("cell().data()",function(a){var b=this.context,c=this[0];if(a===k)return b.length&&c.length?B(b[0],c[0].row,c[0].column):k;jb(b[0],c[0].row,c[0].column,a);ca(b[0],c[0].row,"data",c[0].column);return this});p("order()",function(a,b){var c=this.context;if(a===k)return 0!==
+c.length?c[0].aaSorting:k;"number"===typeof a?a=[[a,b]]:a.length&&!h.isArray(a[0])&&(a=Array.prototype.slice.call(arguments));return this.iterator("table",function(b){b.aaSorting=a.slice()})});p("order.listener()",function(a,b,c){return this.iterator("table",function(d){Ma(d,a,b,c)})});p("order.fixed()",function(a){if(!a){var b=this.context,b=b.length?b[0].aaSortingFixed:k;return h.isArray(b)?{pre:b}:b}return this.iterator("table",function(b){b.aaSortingFixed=h.extend(!0,{},a)})});p(["columns().order()",
+"column().order()"],function(a){var b=this;return this.iterator("table",function(c,d){var e=[];h.each(b[d],function(b,c){e.push([c,a])});c.aaSorting=e})});p("search()",function(a,b,c,d){var e=this.context;return a===k?0!==e.length?e[0].oPreviousSearch.sSearch:k:this.iterator("table",function(e){e.oFeatures.bFilter&&fa(e,h.extend({},e.oPreviousSearch,{sSearch:a+"",bRegex:null===b?!1:b,bSmart:null===c?!0:c,bCaseInsensitive:null===d?!0:d}),1)})});s("columns().search()","column().search()",function(a,
+b,c,d){return this.iterator("column",function(e,f){var g=e.aoPreSearchCols;if(a===k)return g[f].sSearch;e.oFeatures.bFilter&&(h.extend(g[f],{sSearch:a+"",bRegex:null===b?!1:b,bSmart:null===c?!0:c,bCaseInsensitive:null===d?!0:d}),fa(e,e.oPreviousSearch,1))})});p("state()",function(){return this.context.length?this.context[0].oSavedState:null});p("state.clear()",function(){return this.iterator("table",function(a){a.fnStateSaveCallback.call(a.oInstance,a,{})})});p("state.loaded()",function(){return this.context.length?
+this.context[0].oLoadedState:null});p("state.save()",function(){return this.iterator("table",function(a){wa(a)})});m.versionCheck=m.fnVersionCheck=function(a){for(var b=m.version.split("."),a=a.split("."),c,d,e=0,f=a.length;e<f;e++)if(c=parseInt(b[e],10)||0,d=parseInt(a[e],10)||0,c!==d)return c>d;return!0};m.isDataTable=m.fnIsDataTable=function(a){var b=h(a).get(0),c=!1;h.each(m.settings,function(a,e){var f=e.nScrollHead?h("table",e.nScrollHead)[0]:null,g=e.nScrollFoot?h("table",e.nScrollFoot)[0]:
+null;if(e.nTable===b||f===b||g===b)c=!0});return c};m.tables=m.fnTables=function(a){var b=!1;h.isPlainObject(a)&&(b=a.api,a=a.visible);var c=h.map(m.settings,function(b){if(!a||a&&h(b.nTable).is(":visible"))return b.nTable});return b?new r(c):c};m.camelToHungarian=K;p("$()",function(a,b){var c=this.rows(b).nodes(),c=h(c);return h([].concat(c.filter(a).toArray(),c.find(a).toArray()))});h.each(["on","one","off"],function(a,b){p(b+"()",function(){var a=Array.prototype.slice.call(arguments);a[0].match(/\.dt\b/)||
+(a[0]+=".dt");var d=h(this.tables().nodes());d[b].apply(d,a);return this})});p("clear()",function(){return this.iterator("table",function(a){na(a)})});p("settings()",function(){return new r(this.context,this.context)});p("init()",function(){var a=this.context;return a.length?a[0].oInit:null});p("data()",function(){return this.iterator("table",function(a){return G(a.aoData,"_aData")}).flatten()});p("destroy()",function(a){a=a||!1;return this.iterator("table",function(b){var c=b.nTableWrapper.parentNode,
+d=b.oClasses,e=b.nTable,f=b.nTBody,g=b.nTHead,j=b.nTFoot,i=h(e),f=h(f),k=h(b.nTableWrapper),l=h.map(b.aoData,function(a){return a.nTr}),p;b.bDestroying=!0;u(b,"aoDestroyCallback","destroy",[b]);a||(new r(b)).columns().visible(!0);k.unbind(".DT").find(":not(tbody *)").unbind(".DT");h(D).unbind(".DT-"+b.sInstance);e!=g.parentNode&&(i.children("thead").detach(),i.append(g));j&&e!=j.parentNode&&(i.children("tfoot").detach(),i.append(j));b.aaSorting=[];b.aaSortingFixed=[];va(b);h(l).removeClass(b.asStripeClasses.join(" "));
+h("th, td",g).removeClass(d.sSortable+" "+d.sSortableAsc+" "+d.sSortableDesc+" "+d.sSortableNone);b.bJUI&&(h("th span."+d.sSortIcon+", td span."+d.sSortIcon,g).detach(),h("th, td",g).each(function(){var a=h("div."+d.sSortJUIWrapper,this);h(this).append(a.contents());a.detach()}));f.children().detach();f.append(l);g=a?"remove":"detach";i[g]();k[g]();!a&&c&&(c.insertBefore(e,b.nTableReinsertBefore),i.css("width",b.sDestroyWidth).removeClass(d.sTable),(p=b.asDestroyStripes.length)&&f.children().each(function(a){h(this).addClass(b.asDestroyStripes[a%
+p])}));c=h.inArray(b,m.settings);-1!==c&&m.settings.splice(c,1)})});h.each(["column","row","cell"],function(a,b){p(b+"s().every()",function(a){var d=this.selector.opts,e=this;return this.iterator(b,function(f,g,h,i,n){a.call(e[b](g,"cell"===b?h:d,"cell"===b?d:k),g,h,i,n)})})});p("i18n()",function(a,b,c){var d=this.context[0],a=Q(a)(d.oLanguage);a===k&&(a=b);c!==k&&h.isPlainObject(a)&&(a=a[c]!==k?a[c]:a._);return a.replace("%d",c)});m.version="1.10.12";m.settings=[];m.models={};m.models.oSearch={bCaseInsensitive:!0,
+sSearch:"",bRegex:!1,bSmart:!0};m.models.oRow={nTr:null,anCells:null,_aData:[],_aSortData:null,_aFilterData:null,_sFilterRow:null,_sRowStripe:"",src:null,idx:-1};m.models.oColumn={idx:null,aDataSort:null,asSorting:null,bSearchable:null,bSortable:null,bVisible:null,_sManualType:null,_bAttrSrc:!1,fnCreatedCell:null,fnGetData:null,fnSetData:null,mData:null,mRender:null,nTh:null,nTf:null,sClass:null,sContentPadding:null,sDefaultContent:null,sName:null,sSortDataType:"std",sSortingClass:null,sSortingClassJUI:null,
+sTitle:null,sType:null,sWidth:null,sWidthOrig:null};m.defaults={aaData:null,aaSorting:[[0,"asc"]],aaSortingFixed:[],ajax:null,aLengthMenu:[10,25,50,100],aoColumns:null,aoColumnDefs:null,aoSearchCols:[],asStripeClasses:null,bAutoWidth:!0,bDeferRender:!1,bDestroy:!1,bFilter:!0,bInfo:!0,bJQueryUI:!1,bLengthChange:!0,bPaginate:!0,bProcessing:!1,bRetrieve:!1,bScrollCollapse:!1,bServerSide:!1,bSort:!0,bSortMulti:!0,bSortCellsTop:!1,bSortClasses:!0,bStateSave:!1,fnCreatedRow:null,fnDrawCallback:null,fnFooterCallback:null,
+fnFormatNumber:function(a){return a.toString().replace(/\B(?=(\d{3})+(?!\d))/g,this.oLanguage.sThousands)},fnHeaderCallback:null,fnInfoCallback:null,fnInitComplete:null,fnPreDrawCallback:null,fnRowCallback:null,fnServerData:null,fnServerParams:null,fnStateLoadCallback:function(a){try{return JSON.parse((-1===a.iStateDuration?sessionStorage:localStorage).getItem("DataTables_"+a.sInstance+"_"+location.pathname))}catch(b){}},fnStateLoadParams:null,fnStateLoaded:null,fnStateSaveCallback:function(a,b){try{(-1===
+a.iStateDuration?sessionStorage:localStorage).setItem("DataTables_"+a.sInstance+"_"+location.pathname,JSON.stringify(b))}catch(c){}},fnStateSaveParams:null,iStateDuration:7200,iDeferLoading:null,iDisplayLength:10,iDisplayStart:0,iTabIndex:0,oClasses:{},oLanguage:{oAria:{sSortAscending:": activate to sort column ascending",sSortDescending:": activate to sort column descending"},oPaginate:{sFirst:"First",sLast:"Last",sNext:"Next",sPrevious:"Previous"},sEmptyTable:"No data available in table",sInfo:"Showing _START_ to _END_ of _TOTAL_ entries",
+sInfoEmpty:"Showing 0 to 0 of 0 entries",sInfoFiltered:"(filtered from _MAX_ total entries)",sInfoPostFix:"",sDecimal:"",sThousands:",",sLengthMenu:"Show _MENU_ entries",sLoadingRecords:"Loading...",sProcessing:"Processing...",sSearch:"Search:",sSearchPlaceholder:"",sUrl:"",sZeroRecords:"No matching records found"},oSearch:h.extend({},m.models.oSearch),sAjaxDataProp:"data",sAjaxSource:null,sDom:"lfrtip",searchDelay:null,sPaginationType:"simple_numbers",sScrollX:"",sScrollXInner:"",sScrollY:"",sServerMethod:"GET",
+renderer:null,rowId:"DT_RowId"};X(m.defaults);m.defaults.column={aDataSort:null,iDataSort:-1,asSorting:["asc","desc"],bSearchable:!0,bSortable:!0,bVisible:!0,fnCreatedCell:null,mData:null,mRender:null,sCellType:"td",sClass:"",sContentPadding:"",sDefaultContent:null,sName:"",sSortDataType:"std",sTitle:null,sType:null,sWidth:null};X(m.defaults.column);m.models.oSettings={oFeatures:{bAutoWidth:null,bDeferRender:null,bFilter:null,bInfo:null,bLengthChange:null,bPaginate:null,bProcessing:null,bServerSide:null,
+bSort:null,bSortMulti:null,bSortClasses:null,bStateSave:null},oScroll:{bCollapse:null,iBarWidth:0,sX:null,sXInner:null,sY:null},oLanguage:{fnInfoCallback:null},oBrowser:{bScrollOversize:!1,bScrollbarLeft:!1,bBounding:!1,barWidth:0},ajax:null,aanFeatures:[],aoData:[],aiDisplay:[],aiDisplayMaster:[],aIds:{},aoColumns:[],aoHeader:[],aoFooter:[],oPreviousSearch:{},aoPreSearchCols:[],aaSorting:null,aaSortingFixed:[],asStripeClasses:null,asDestroyStripes:[],sDestroyWidth:0,aoRowCallback:[],aoHeaderCallback:[],
+aoFooterCallback:[],aoDrawCallback:[],aoRowCreatedCallback:[],aoPreDrawCallback:[],aoInitComplete:[],aoStateSaveParams:[],aoStateLoadParams:[],aoStateLoaded:[],sTableId:"",nTable:null,nTHead:null,nTFoot:null,nTBody:null,nTableWrapper:null,bDeferLoading:!1,bInitialised:!1,aoOpenRows:[],sDom:null,searchDelay:null,sPaginationType:"two_button",iStateDuration:0,aoStateSave:[],aoStateLoad:[],oSavedState:null,oLoadedState:null,sAjaxSource:null,sAjaxDataProp:null,bAjaxDataGet:!0,jqXHR:null,json:k,oAjaxData:k,
+fnServerData:null,aoServerParams:[],sServerMethod:null,fnFormatNumber:null,aLengthMenu:null,iDraw:0,bDrawing:!1,iDrawError:-1,_iDisplayLength:10,_iDisplayStart:0,_iRecordsTotal:0,_iRecordsDisplay:0,bJUI:null,oClasses:{},bFiltered:!1,bSorted:!1,bSortCellsTop:null,oInit:null,aoDestroyCallback:[],fnRecordsTotal:function(){return"ssp"==y(this)?1*this._iRecordsTotal:this.aiDisplayMaster.length},fnRecordsDisplay:function(){return"ssp"==y(this)?1*this._iRecordsDisplay:this.aiDisplay.length},fnDisplayEnd:function(){var a=
+this._iDisplayLength,b=this._iDisplayStart,c=b+a,d=this.aiDisplay.length,e=this.oFeatures,f=e.bPaginate;return e.bServerSide?!1===f||-1===a?b+d:Math.min(b+a,this._iRecordsDisplay):!f||c>d||-1===a?d:c},oInstance:null,sInstance:null,iTabIndex:0,nScrollHead:null,nScrollFoot:null,aLastSort:[],oPlugins:{},rowIdFn:null,rowId:null};m.ext=v={buttons:{},classes:{},builder:"-source-",errMode:"alert",feature:[],search:[],selector:{cell:[],column:[],row:[]},internal:{},legacy:{ajax:null},pager:{},renderer:{pageButton:{},
+header:{}},order:{},type:{detect:[],search:{},order:{}},_unique:0,fnVersionCheck:m.fnVersionCheck,iApiIndex:0,oJUIClasses:{},sVersion:m.version};h.extend(v,{afnFiltering:v.search,aTypes:v.type.detect,ofnSearch:v.type.search,oSort:v.type.order,afnSortData:v.order,aoFeatures:v.feature,oApi:v.internal,oStdClasses:v.classes,oPagination:v.pager});h.extend(m.ext.classes,{sTable:"dataTable",sNoFooter:"no-footer",sPageButton:"paginate_button",sPageButtonActive:"current",sPageButtonDisabled:"disabled",sStripeOdd:"odd",
+sStripeEven:"even",sRowEmpty:"dataTables_empty",sWrapper:"dataTables_wrapper",sFilter:"dataTables_filter",sInfo:"dataTables_info",sPaging:"dataTables_paginate paging_",sLength:"dataTables_length",sProcessing:"dataTables_processing",sSortAsc:"sorting_asc",sSortDesc:"sorting_desc",sSortable:"sorting",sSortableAsc:"sorting_asc_disabled",sSortableDesc:"sorting_desc_disabled",sSortableNone:"sorting_disabled",sSortColumn:"sorting_",sFilterInput:"",sLengthSelect:"",sScrollWrapper:"dataTables_scroll",sScrollHead:"dataTables_scrollHead",
+sScrollHeadInner:"dataTables_scrollHeadInner",sScrollBody:"dataTables_scrollBody",sScrollFoot:"dataTables_scrollFoot",sScrollFootInner:"dataTables_scrollFootInner",sHeaderTH:"",sFooterTH:"",sSortJUIAsc:"",sSortJUIDesc:"",sSortJUI:"",sSortJUIAscAllowed:"",sSortJUIDescAllowed:"",sSortJUIWrapper:"",sSortIcon:"",sJUIHeader:"",sJUIFooter:""});var Ca="",Ca="",H=Ca+"ui-state-default",ia=Ca+"css_right ui-icon ui-icon-",Xb=Ca+"fg-toolbar ui-toolbar ui-widget-header ui-helper-clearfix";h.extend(m.ext.oJUIClasses,
+m.ext.classes,{sPageButton:"fg-button ui-button "+H,sPageButtonActive:"ui-state-disabled",sPageButtonDisabled:"ui-state-disabled",sPaging:"dataTables_paginate fg-buttonset ui-buttonset fg-buttonset-multi ui-buttonset-multi paging_",sSortAsc:H+" sorting_asc",sSortDesc:H+" sorting_desc",sSortable:H+" sorting",sSortableAsc:H+" sorting_asc_disabled",sSortableDesc:H+" sorting_desc_disabled",sSortableNone:H+" sorting_disabled",sSortJUIAsc:ia+"triangle-1-n",sSortJUIDesc:ia+"triangle-1-s",sSortJUI:ia+"carat-2-n-s",
+sSortJUIAscAllowed:ia+"carat-1-n",sSortJUIDescAllowed:ia+"carat-1-s",sSortJUIWrapper:"DataTables_sort_wrapper",sSortIcon:"DataTables_sort_icon",sScrollHead:"dataTables_scrollHead "+H,sScrollFoot:"dataTables_scrollFoot "+H,sHeaderTH:H,sFooterTH:H,sJUIHeader:Xb+" ui-corner-tl ui-corner-tr",sJUIFooter:Xb+" ui-corner-bl ui-corner-br"});var Mb=m.ext.pager;h.extend(Mb,{simple:function(){return["previous","next"]},full:function(){return["first","previous","next","last"]},numbers:function(a,b){return[ya(a,
+b)]},simple_numbers:function(a,b){return["previous",ya(a,b),"next"]},full_numbers:function(a,b){return["first","previous",ya(a,b),"next","last"]},_numbers:ya,numbers_length:7});h.extend(!0,m.ext.renderer,{pageButton:{_:function(a,b,c,d,e,f){var g=a.oClasses,j=a.oLanguage.oPaginate,i=a.oLanguage.oAria.paginate||{},k,l,m=0,p=function(b,d){var o,r,u,s,v=function(b){Ta(a,b.data.action,true)};o=0;for(r=d.length;o<r;o++){s=d[o];if(h.isArray(s)){u=h("<"+(s.DT_el||"div")+"/>").appendTo(b);p(u,s)}else{k=null;
+l="";switch(s){case "ellipsis":b.append('<span class="ellipsis">&#x2026;</span>');break;case "first":k=j.sFirst;l=s+(e>0?"":" "+g.sPageButtonDisabled);break;case "previous":k=j.sPrevious;l=s+(e>0?"":" "+g.sPageButtonDisabled);break;case "next":k=j.sNext;l=s+(e<f-1?"":" "+g.sPageButtonDisabled);break;case "last":k=j.sLast;l=s+(e<f-1?"":" "+g.sPageButtonDisabled);break;default:k=s+1;l=e===s?g.sPageButtonActive:""}if(k!==null){u=h("<a>",{"class":g.sPageButton+" "+l,"aria-controls":a.sTableId,"aria-label":i[s],
+"data-dt-idx":m,tabindex:a.iTabIndex,id:c===0&&typeof s==="string"?a.sTableId+"_"+s:null}).html(k).appendTo(b);Wa(u,{action:s},v);m++}}}},r;try{r=h(b).find(I.activeElement).data("dt-idx")}catch(o){}p(h(b).empty(),d);r&&h(b).find("[data-dt-idx="+r+"]").focus()}}});h.extend(m.ext.type.detect,[function(a,b){var c=b.oLanguage.sDecimal;return Za(a,c)?"num"+c:null},function(a){if(a&&!(a instanceof Date)&&(!ac.test(a)||!bc.test(a)))return null;var b=Date.parse(a);return null!==b&&!isNaN(b)||M(a)?"date":
+null},function(a,b){var c=b.oLanguage.sDecimal;return Za(a,c,!0)?"num-fmt"+c:null},function(a,b){var c=b.oLanguage.sDecimal;return Rb(a,c)?"html-num"+c:null},function(a,b){var c=b.oLanguage.sDecimal;return Rb(a,c,!0)?"html-num-fmt"+c:null},function(a){return M(a)||"string"===typeof a&&-1!==a.indexOf("<")?"html":null}]);h.extend(m.ext.type.search,{html:function(a){return M(a)?a:"string"===typeof a?a.replace(Ob," ").replace(Aa,""):""},string:function(a){return M(a)?a:"string"===typeof a?a.replace(Ob,
+" "):a}});var za=function(a,b,c,d){if(0!==a&&(!a||"-"===a))return-Infinity;b&&(a=Qb(a,b));a.replace&&(c&&(a=a.replace(c,"")),d&&(a=a.replace(d,"")));return 1*a};h.extend(v.type.order,{"date-pre":function(a){return Date.parse(a)||0},"html-pre":function(a){return M(a)?"":a.replace?a.replace(/<.*?>/g,"").toLowerCase():a+""},"string-pre":function(a){return M(a)?"":"string"===typeof a?a.toLowerCase():!a.toString?"":a.toString()},"string-asc":function(a,b){return a<b?-1:a>b?1:0},"string-desc":function(a,
+b){return a<b?1:a>b?-1:0}});db("");h.extend(!0,m.ext.renderer,{header:{_:function(a,b,c,d){h(a.nTable).on("order.dt.DT",function(e,f,g,h){if(a===f){e=c.idx;b.removeClass(c.sSortingClass+" "+d.sSortAsc+" "+d.sSortDesc).addClass(h[e]=="asc"?d.sSortAsc:h[e]=="desc"?d.sSortDesc:c.sSortingClass)}})},jqueryui:function(a,b,c,d){h("<div/>").addClass(d.sSortJUIWrapper).append(b.contents()).append(h("<span/>").addClass(d.sSortIcon+" "+c.sSortingClassJUI)).appendTo(b);h(a.nTable).on("order.dt.DT",function(e,
+f,g,h){if(a===f){e=c.idx;b.removeClass(d.sSortAsc+" "+d.sSortDesc).addClass(h[e]=="asc"?d.sSortAsc:h[e]=="desc"?d.sSortDesc:c.sSortingClass);b.find("span."+d.sSortIcon).removeClass(d.sSortJUIAsc+" "+d.sSortJUIDesc+" "+d.sSortJUI+" "+d.sSortJUIAscAllowed+" "+d.sSortJUIDescAllowed).addClass(h[e]=="asc"?d.sSortJUIAsc:h[e]=="desc"?d.sSortJUIDesc:c.sSortingClassJUI)}})}}});var Yb=function(a){return"string"===typeof a?a.replace(/</g,"&lt;").replace(/>/g,"&gt;").replace(/"/g,"&quot;"):a};m.render={number:function(a,
+b,c,d,e){return{display:function(f){if("number"!==typeof f&&"string"!==typeof f)return f;var g=0>f?"-":"",h=parseFloat(f);if(isNaN(h))return Yb(f);f=Math.abs(h);h=parseInt(f,10);f=c?b+(f-h).toFixed(c).substring(2):"";return g+(d||"")+h.toString().replace(/\B(?=(\d{3})+(?!\d))/g,a)+f+(e||"")}}},text:function(){return{display:Yb}}};h.extend(m.ext.internal,{_fnExternApiFunc:Nb,_fnBuildAjax:ra,_fnAjaxUpdate:lb,_fnAjaxParameters:ub,_fnAjaxUpdateDraw:vb,_fnAjaxDataSrc:sa,_fnAddColumn:Ea,_fnColumnOptions:ja,
+_fnAdjustColumnSizing:Y,_fnVisibleToColumnIndex:Z,_fnColumnIndexToVisible:$,_fnVisbleColumns:aa,_fnGetColumns:la,_fnColumnTypes:Ga,_fnApplyColumnDefs:ib,_fnHungarianMap:X,_fnCamelToHungarian:K,_fnLanguageCompat:Da,_fnBrowserDetect:gb,_fnAddData:N,_fnAddTr:ma,_fnNodeToDataIndex:function(a,b){return b._DT_RowIndex!==k?b._DT_RowIndex:null},_fnNodeToColumnIndex:function(a,b,c){return h.inArray(c,a.aoData[b].anCells)},_fnGetCellData:B,_fnSetCellData:jb,_fnSplitObjNotation:Ja,_fnGetObjectDataFn:Q,_fnSetObjectDataFn:R,
+_fnGetDataMaster:Ka,_fnClearTable:na,_fnDeleteIndex:oa,_fnInvalidate:ca,_fnGetRowElements:Ia,_fnCreateTr:Ha,_fnBuildHead:kb,_fnDrawHead:ea,_fnDraw:O,_fnReDraw:T,_fnAddOptionsHtml:nb,_fnDetectHeader:da,_fnGetUniqueThs:qa,_fnFeatureHtmlFilter:pb,_fnFilterComplete:fa,_fnFilterCustom:yb,_fnFilterColumn:xb,_fnFilter:wb,_fnFilterCreateSearch:Pa,_fnEscapeRegex:Qa,_fnFilterData:zb,_fnFeatureHtmlInfo:sb,_fnUpdateInfo:Cb,_fnInfoMacros:Db,_fnInitialise:ga,_fnInitComplete:ta,_fnLengthChange:Ra,_fnFeatureHtmlLength:ob,
+_fnFeatureHtmlPaginate:tb,_fnPageChange:Ta,_fnFeatureHtmlProcessing:qb,_fnProcessingDisplay:C,_fnFeatureHtmlTable:rb,_fnScrollDraw:ka,_fnApplyToChildren:J,_fnCalculateColumnWidths:Fa,_fnThrottle:Oa,_fnConvertToWidth:Fb,_fnGetWidestNode:Gb,_fnGetMaxLenString:Hb,_fnStringToCss:x,_fnSortFlatten:V,_fnSort:mb,_fnSortAria:Jb,_fnSortListener:Va,_fnSortAttachListener:Ma,_fnSortingClasses:va,_fnSortData:Ib,_fnSaveState:wa,_fnLoadState:Kb,_fnSettingsFromNode:xa,_fnLog:L,_fnMap:E,_fnBindAction:Wa,_fnCallbackReg:z,
+_fnCallbackFire:u,_fnLengthOverflow:Sa,_fnRenderer:Na,_fnDataSource:y,_fnRowAttributes:La,_fnCalculateEnd:function(){}});h.fn.dataTable=m;m.$=h;h.fn.dataTableSettings=m.settings;h.fn.dataTableExt=m.ext;h.fn.DataTable=function(a){return h(this).dataTable(a).api()};h.each(m,function(a,b){h.fn.DataTable[a]=b});return h.fn.dataTable});

static/master_list.js

@@ -0,0 +1,392 @@
+//////////////////////////////////
+// Centillion Master List
+// Javascript Functions 
+//
+// This file contains javascript functions used by
+// the master_list page of centillion. The master
+// list page uses the search engine as an API to
+// get a list of all documents by type.
+
+
+///////////////////////////////////
+// Process get parameters
+//
+// When the document is loaded, parse the GET params
+// from the URL (everything after the ?).
+//
+// If the "doctype" parameter is in the URL, use it to
+// determine which panel to open automatically.
+
+var initGdocTable = false;
+var initIssuesTable = false;
+var initGhfilesTable = false;
+var initMarkdownTable = false;
+var initEmailthreadsTable = false;
+
+$(document).ready(function() {
+    var url_string = document.location.toString();
+    var url = new URL(url_string);
+    var d = url.searchParams.get("doctype");
+
+    if (d==='gdoc') {
+        load_gdoc_table();
+        var divList = $('div#collapseDrive').addClass('in');
+
+    } else if (d==='emailthread') {
+        load_emailthreads_table();
+        var divList = $('div#collapseThreads').addClass('in');
+
+    } else if (d==='issue') {
+        load_issue_table();
+        var divList = $('div#collapseIssues').addClass('in');
+
+    } else if (d==='ghfile') {
+        load_ghfile_table();
+        var divList = $('div#collapseFiles').addClass('in');
+
+    } else if (d==='markdown') {
+        load_markdown_table();
+        var divList = $('div#collapseMarkdown').addClass('in');
+
+    }
+});
+
+
+
+//////////////////////////////////
+// API-to-Table Functions
+//
+// These functions ask centillion for a list of all documents
+// of a given type, and load the results into an HTML table.
+//
+// The dataTable bootstrap plugin is used to make the tables
+// sortable, searchable, and slick.
+//
+// Sections:
+// ----------
+// Google Drive files
+// Github issues
+// Github files
+// Github markdown
+// Groups.io email threads
+
+// ------------------------
+// Google Drive
+
+function load_gdoc_table(){
+    if(!initGdocTable) {
+        var divList = $('div#collapseDrive').attr('class');
+        if (divList.indexOf('in') !== -1) {
+            console.log('Closing Google Drive master list');
+        } else { 
+            console.log('Opening Google Drive master list');
+
+            $.getJSON("/list/gdoc", function(result){
+
+                var r = new Array(), j = -1, size=result.length;
+                r[++j] = '<thead>'
+                r[++j] = '<tr class="header-row">';
+                r[++j] = '<th width="40%">File Name</th>';
+                r[++j] = '<th width="15%">Owner</th>';
+                r[++j] = '<th width="15%">Type</th>';
+                r[++j] = '<th width="15%">Created</th>';
+                r[++j] = '<th width="15%">Modified</th>';
+                r[++j] = '</tr>';
+                r[++j] = '</thead>'
+                r[++j] = '<tbody>'
+                for (var i=0; i<size; i++){
+                    r[++j] = '<tr><td>';
+                    r[++j] = '<a href="' + result[i]['url'] + '" target="_blank">'
+                    r[++j] = result[i]['title'];
+                    r[++j] = '</a>'
+                    r[++j] = '</td><td>';
+                    r[++j] = result[i]['owner_name'];
+                    r[++j] = '</td><td>';
+                    r[++j] = result[i]['mimetype'];
+                    r[++j] = '</td><td>';
+                    r[++j] = result[i]['created_time'];
+                    r[++j] = '</td><td>';
+                    r[++j] = result[i]['modified_time'];
+                    r[++j] = '</td></tr>';
+                }
+                r[++j] = '</tbody>'
+
+                // Construct names of id tags
+                var doctype = 'gdocs';
+                var idlabel = '#' + doctype + '-master-list';
+                var filtlabel = idlabel + '_filter';
+
+                // Initialize the DataTable
+                $(idlabel).html(r.join(''));
+                $(idlabel).DataTable({
+                    responsive: true,
+                    lengthMenu: [50,100,250,500]
+                });
+
+                // Get the search filter section and search box
+                var searchsec = $(filtlabel).find('label');
+                var searchbox = searchsec.find('input');
+
+                // Replace search filter section text,
+                // then re-add the removed search box
+                searchsec.text('Search Metadata: ');
+                searchsec.append(searchbox);
+
+                initGdocTable = true
+            });
+            console.log('Finished loading Google Drive master list');
+        }
+    }
+}
+
+// ------------------------
+// Github issues
+
+function load_issue_table(){
+    if(!initIssuesTable) {
+        var divList = $('div#collapseIssues').attr('class');
+        if (divList.indexOf('in') !== -1) {
+            console.log('Closing Github issues master list');
+        } else { 
+            console.log('Opening Github issues master list');
+
+            $.getJSON("/list/issue", function(result){
+                var r = new Array(), j = -1, size=result.length;
+                r[++j] = '<thead>'
+                r[++j] = '<tr class="header-row">';
+                r[++j] = '<th width="50%">Issue Name</th>';
+                r[++j] = '<th width="15%">Repository</th>';
+                r[++j] = '<th width="15%">Created</th>';
+                r[++j] = '<th width="15%">Modified</th>';
+                r[++j] = '</tr>';
+                r[++j] = '</thead>'
+                r[++j] = '<tbody>'
+                for (var i=0; i<size; i++){
+                    r[++j] ='<tr><td>';
+                    r[++j] = '<a href="' + result[i]['url'] + '" target="_blank">'
+                    r[++j] = result[i]['title'];
+                    r[++j] = '</a>'
+                    r[++j] = '</td><td>';
+                    r[++j] = '<a href="' + result[i]['repo_url'] + '" target="_blank">'
+                    r[++j] = result[i]['repo_name'];
+                    r[++j] = '</a>'
+                    r[++j] = '</td><td>';
+                    r[++j] = result[i]['created_time'];
+                    r[++j] = '</td><td>';
+                    r[++j] = result[i]['modified_time'];
+                    r[++j] = '</td></tr>';
+                }
+                r[++j] = '</tbody>'
+
+                // Construct names of id tags
+                var doctype = 'issues';
+                var idlabel = '#' + doctype + '-master-list';
+                var filtlabel = idlabel + '_filter';
+
+                // Initialize the DataTable
+                $(idlabel).html(r.join(''));
+                $(idlabel).DataTable({
+                    responsive: true,
+                    lengthMenu: [50,100,250,500]
+                });
+
+                // Get the search filter section and search box
+                var searchsec = $(filtlabel).find('label');
+                var searchbox = searchsec.find('input');
+
+                // Replace search filter section text,
+                // then re-add the removed search box
+                searchsec.text('Search Metadata: ');
+                searchsec.append(searchbox);
+
+                initIssuesTable = true;
+            });
+            console.log('Finished loading Github issues master list');
+        }
+    }
+}
+
+// ------------------------
+// Github files
+
+function load_ghfile_table(){
+    if(!initGhfilesTable) {
+        var divList = $('div#collapseFiles').attr('class');
+        if (divList.indexOf('in') !== -1) {
+            console.log('Closing Github files master list');
+        } else { 
+            console.log('Opening Github files master list');
+
+            $.getJSON("/list/ghfile", function(result){
+                console.log("-----------");
+                console.log(result);
+                var r = new Array(), j = -1, size=result.length;
+                r[++j] = '<thead>'
+                r[++j] = '<tr class="header-row">';
+                r[++j] = '<th width="70%">File Name</th>';
+                r[++j] = '<th width="30%">Repository</th>';
+                r[++j] = '</tr>';
+                r[++j] = '</thead>'
+                r[++j] = '<tbody>'
+                for (var i=0; i<size; i++){
+                    r[++j] ='<tr><td>';
+                    r[++j] = '<a href="' + result[i]['url'] + '" target="_blank">'
+                    r[++j] = result[i]['title'];
+                    r[++j] = '</a>'
+                    r[++j] = '</td><td>';
+                    r[++j] = '<a href="' + result[i]['repo_url'] + '" target="_blank">'
+                    r[++j] = result[i]['repo_name'];
+                    r[++j] = '</a>'
+                    r[++j] = '</td></tr>';
+                }
+                r[++j] = '</tbody>'
+
+                // Construct names of id tags
+                var doctype = 'ghfiles';
+                var idlabel = '#' + doctype + '-master-list';
+                var filtlabel = idlabel + '_filter';
+
+                // Initialize the DataTable
+                $(idlabel).html(r.join(''));
+                $(idlabel).DataTable({
+                    responsive: true,
+                    lengthMenu: [50,100,250,500]
+                });
+
+                // Get the search filter section and search box
+                var searchsec = $(filtlabel).find('label');
+                var searchbox = searchsec.find('input');
+
+                // Replace search filter section text,
+                // then re-add the removed search box
+                searchsec.text('Search Metadata: ');
+                searchsec.append(searchbox);
+
+                initGhfilesTable = true;
+            });
+            console.log('Finished loading Github file list');
+        }
+    }
+}
+
+// ------------------------
+// Github Markdown
+
+function load_markdown_table(){
+    if(!initMarkdownTable) { 
+        var divList = $('div#collapseMarkdown').attr('class');
+        if (divList.indexOf('in') !== -1) {
+            console.log('Closing Github markdown master list');
+        } else { 
+            console.log('Opening Github markdown master list');
+
+            $.getJSON("/list/markdown", function(result){
+                var r = new Array(), j = -1, size=result.length;
+                r[++j] = '<thead>'
+                r[++j] = '<tr class="header-row">';
+                r[++j] = '<th width="70%">Markdown File Name</th>';
+                r[++j] = '<th width="30%">Repository</th>';
+                r[++j] = '</tr>';
+                r[++j] = '</thead>'
+                r[++j] = '<tbody>'
+                for (var i=0; i<size; i++){
+                    r[++j] ='<tr><td>';
+                    r[++j] = '<a href="' + result[i]['url'] + '" target="_blank">'
+                    r[++j] = result[i]['title'];
+                    r[++j] = '</a>'
+                    r[++j] = '</td><td>';
+                    r[++j] = '<a href="' + result[i]['repo_url'] + '" target="_blank">'
+                    r[++j] = result[i]['repo_name'];
+                    r[++j] = '</a>'
+                    r[++j] = '</td></tr>';
+                }
+                r[++j] = '</tbody>'
+
+                // Construct names of id tags
+                var doctype = 'markdown';
+                var idlabel = '#' + doctype + '-master-list';
+                var filtlabel = idlabel + '_filter';
+
+                // Initialize the DataTable
+                $(idlabel).html(r.join(''));
+                $(idlabel).DataTable({
+                    responsive: true,
+                    lengthMenu: [50,100,250,500]
+                });
+
+                // Get the search filter section and search box
+                var searchsec = $(filtlabel).find('label');
+                var searchbox = searchsec.find('input');
+
+                // Replace search filter section text,
+                // then re-add the removed search box
+                searchsec.text('Search Metadata: ');
+                searchsec.append(searchbox);
+
+                initMarkdownTable = true;
+            });
+            console.log('Finished loading Markdown list');
+        }
+    }
+}
+
+
+// ------------------------
+// Groups.io Email Threads
+
+function load_emailthreads_table(){
+    if(!initEmailthreadsTable) { 
+        var divList = $('div#collapseThreads').attr('class');
+        if (divList.indexOf('in') !== -1) {
+            console.log('Closing Groups.io email threads master list');
+        } else { 
+            console.log('Opening Groups.io email threads master list');
+    
+            $.getJSON("/list/emailthread", function(result){
+                var r = new Array(), j = -1, size=result.length;
+                r[++j] = '<thead>'
+                r[++j] = '<tr class="header-row">';
+                r[++j] = '<th width="70%">Topic</th>';
+                r[++j] = '<th width="30%">Started By</th>';
+                r[++j] = '</tr>';
+                r[++j] = '</thead>'
+                r[++j] = '<tbody>'
+                for (var i=0; i<size; i++){
+                    r[++j] ='<tr><td>';
+                    r[++j] = '<a href="' + result[i]['url'] + '" target="_blank">'
+                    r[++j] = result[i]['title'];
+                    r[++j] = '</a>'
+                    r[++j] = '</td><td>';
+                    r[++j] = result[i]['owner_name'];
+                    r[++j] = '</td></tr>';
+                }
+                r[++j] = '</tbody>'
+
+                // Construct names of id tags
+                var doctype = 'emailthreads';
+                var idlabel = '#' + doctype + '-master-list';
+                var filtlabel = idlabel + '_filter';
+
+                // Initialize the DataTable
+                $(idlabel).html(r.join(''));
+                $(idlabel).DataTable({
+                    responsive: true,
+                    lengthMenu: [50,100,250,500]
+                });
+
+                // Get the search filter section and search box
+                var searchsec = $(filtlabel).find('label');
+                var searchbox = searchsec.find('input');
+
+                // Replace search filter section text,
+                // then re-add the removed search box
+                searchsec.text('Search Metadata: ');
+                searchsec.append(searchbox);
+
+                initEmailthreadsTable = true;
+            });
+            console.log('Finished loading Groups.io email threads list');
+        }
+    }
+}
+

static/search_list.js

@@ -0,0 +1,39 @@
+//////////////////////////////////
+// Centillion Search Results Listing
+// Javascript Functions 
+//
+// This file contains javascript functions used by
+// the search results page centillion.
+
+
+
+//////////////////////////////////
+// Results-to-DataTable Functions
+//
+// These functions post-process the table of search results
+// and make it into a dataTable.
+//
+// The dataTable bootstrap plugin is used to make the tables
+// sortable, searchable, and slick.
+
+
+$(document).ready(function() {
+    // Construct names of id tags
+    table_id = "#search-results";
+
+    // Initialize the DataTable
+    $(table_id).DataTable({
+        responsive: true,
+        searching: false,
+        order: [[0,'desc']],
+        aoColumnDefs: [
+            { bSortable: false,
+              aTargets : [2]
+            }
+        ],
+        lengthMenu: [50,100,250,500]
+    });
+
+    console.log('Finished loading search results list');
+});
+

static/style.css

@@ -1,3 +1,33 @@
+td#search-results-score-col,
+td#search-results-type-col {
+    width: 100px;
+}
+
+div.container {
+    width: 90%;
+}
+
+/* control panel button width */
+.btn-reindex-type, .btn-reindex-all {
+    width: 350px;
+}
+
+/* landing page github button */
+#github-button {
+    display:inline-block;
+    font-size: 20px;
+    line-height: 40px;
+    padding: 10px 40px 10px 40px;
+    text-align: center;
+}
+
+/* search button */
+#the-big-one {
+    margin-top: 10px;
+    margin-bottom: 10px;
+}
+
+/* badges for number of docs indexed */
 span.badge {
     vertical-align: text-bottom;
 }
@@ -7,6 +37,10 @@ a.badgelinks, a.badgelinks:hover {
     text-decoration: none;
 }
 
+h2.masterlist-header a {
+    text-decoration: none;
+}
+
 div.list-group {
     border: 1px solid rgba(86,61,124,.2);
 }

templates/403.html

@@ -0,0 +1,25 @@
+{% extends "layout.html" %}
+{% block body %}
+
+<div class="container">
+    <div class="row">
+        <p>&nbsp;</p>
+    </div>
+    <div class="row">
+
+        <div class="col-xs-12">
+            <center>
+                <h2>403: Permission Denied
+                </h2>
+                <p>You don't seem to be a member of the proper Github team or organization.
+                Please contact <a href="mailto:dcppc-inbox@gmail.com">dcppc-inbox@gmail.com</a>
+                if you believe you are seeing this message in error.
+                </p>
+            </center>
+        </div>
+
+    </div>
+</div>
+
+{% endblock %}
+

templates/404.html

@@ -0,0 +1,21 @@
+{% extends "layout.html" %}
+{% block body %}
+
+<div class="container">
+    <div class="row">
+
+        <div class="col-xs-12">
+            <center>
+                <h2>404: Not Found
+                </h2>
+                <p>You have requested a page that does not exist.
+                Try clicking the centillion logo to go back home.
+                </p>
+            </center>
+        </div>
+
+    </div>
+</div>
+
+{% endblock %}
+

templates/controlpanel.html

@@ -1,52 +1,6 @@
 {% extends "layout.html" %}
 {% block body %}
 
-{% with messages = get_flashed_messages() %}
-{% if messages %}
-<div class="container">
-    <div class="alert alert-success alert-dismissible">
-        <a href="#" class="close" data-dismiss="alert" aria-label="close">&times;</a>
-        <ul class=flashes>
-            {% for message in messages %}
-            <li>{{ message }}</li>
-            {% endfor %}
-        </ul>
-    </div>
-</div>
-{% endif %}
-{% endwith %}
-
-<div class="container">
-
-    <div class="row">
-        <div class="col-md-12">
-            <center>
-                <a href="{{ url_for('search')}}?query=&fields=">
-                <img src="{{ url_for('static', filename='centillion_white.png') }}">
-                </a>
-                {% if config['TAGLINE'] %}
-                    <h2><a href="{{ url_for('search')}}?query=&fields=">
-                        {{config['TAGLINE']}}
-                    </a></h2>
-                {% endif %}
-            </center>
-        </div>
-    </div>
-    
-    {% if config['zzzTAGLINE'] %}
-    <div class="row">
-        <div class="col12sm">
-            <center>
-                <h2><a href="{{ url_for('search')}}?query=&fields=">
-                    {{config['TAGLINE']}}
-                </a></h2>
-            </center>
-        </div>
-    </div>
-    {% endif %}
-
-</div>
-
 <hr />
 
 <div class="container">
@@ -66,9 +20,9 @@
                         <div class="col-md-12">
                             <p class="panel-text">Re-index <i>every</i> document in the
                             remote collection in the search index. <b>Warning: this operation may take a while.</b>
-                            <p/> <p>
-                            <a href="{{ url_for('update_index') }}" class="btn btn-large btn-danger">Update Main Index</a>
-                            <p/> 
+                            </p>
+                            <p><a id="re-index-main" href="{{ url_for('update_index',run_which='all') }}" class="btn btn-large btn-danger btn-reindex-all">Update Main Index</a>
+                            </p> 
                         </div>
                     </div>
                 </div>
@@ -76,22 +30,29 @@
             </div>
         </div>
 
-        {# update diff search index #}
+
+
+        {# update search index by type #}
         <div class="panel panel-danger">
             <div class="panel-heading">
                 <h3 class="panel-title">
-                    Update Diff Search Index
+                    Update Search Index by Type
                 </h3>
             </div>
             <div class="panel-body">
                 <div class="container-fluid">
                     <div class="row">
                         <div class="col-md-12">
-                            <p class="panel-text">Diff search index only re-indexes documents created after the last
-                            search index update. <b>Not currently implemented.</b>
-                            <p/> <p>
-                            <a href="#" class="btn btn-large disabled btn-danger">Update Diff Index</a>
-                            <p/> 
+                            <p class="panel-text">Re-index individual document types in the search index.
+                            </p>
+                            <p><a href="{{ url_for('update_index',run_which='gdocs') }}"        class="btn btn-large btn-danger btn-reindex-type">Update Google Drive Index</a>
+                            </p>  
+                            <p><a href="{{ url_for('update_index',run_which='ghfiles') }}"      class="btn btn-large btn-danger btn-reindex-type">Update Github Files Index</a>
+                            </p>  
+                            <p><a href="{{ url_for('update_index',run_which='issues') }}"       class="btn btn-large btn-danger btn-reindex-type">Update Github Issues Index</a>
+                            </p>  
+                            <p><a href="{{ url_for('update_index',run_which='emailthreads') }}" class="btn btn-large btn-danger btn-reindex-type">Update Groups.io Email Threads Index</a>
+                            </p> 
                         </div>
                     </div>
                 </div>

templates/landing.html

@@ -0,0 +1,22 @@
+{% extends "layout.html" %}
+{% block body %}
+<div class="container">
+    <div class="row">
+
+        <div class="col-xs-12">
+
+            <center>
+                <p class="lead">To begin searching the Data Commons, you must first sign in with Github.
+                </p>
+                <a href="/log_in" type="button" id="github-button" class="btn btn-default">
+                <i class="fa fa-github fa-2x"></i> Sign in with Github
+                </a>
+            </center>
+
+        </div>
+
+    </div>
+</div>
+{% endblock %}
+
+

templates/layout.html

@@ -1,12 +1,74 @@
 <!doctype html>
-<title>Markdown Search</title>
+<title>Centillion Search Engine</title>
+
 <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='style.css') }}">
 <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='github-markdown.css') }}">
 <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='bootstrap.min.css') }}">
+<link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='font-awesome.min.css') }}">
 
 <script src="{{ url_for('static', filename='jquery.min.js') }}"></script>
 <script src="{{ url_for('static', filename='bootstrap.min.js') }}"></script>
+<script src="{{ url_for('static', filename='master_list.js') }}"></script>
+<script src="{{ url_for('static', filename='search_list.js') }}"></script>
+
+
+{# ########## dataTables plugin ############ #}
+
+<link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='dataTables.bootstrap.min.css') }}">
+<link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='dataTables.responsive.css') }}">
+
+<script src="{{ url_for('static', filename='jquery.dataTables.min.js') }}"></script>
+<script src="{{ url_for('static', filename='dataTables.bootstrap.min.js') }}"></script>
+<script src="{{ url_for('static', filename='dataTables.responsive.js') }}"></script>
+
+
+
+{# ########## github fork corner ############ #}
 
 <div>
+
+    {% with messages = get_flashed_messages() %}
+    {% if messages %}
+    <div class="container">
+        <div class="alert alert-success alert-dismissible">
+            <a href="#" class="close" data-dismiss="alert" aria-label="close">&times;</a>
+            <ul class=flashes>
+                {% for message in messages %}
+                <li>{{ message }}</li>
+                {% endfor %}
+            </ul>
+        </div>
+    </div>
+    {% endif %}
+    {% endwith %}
+
+    <div class="container">
+    
+        {#
+        banner image
+        #}
+        <div class="row">
+            <div class="col12sm">
+                <center>
+                    <a href="{{ url_for('search')}}?query=&fields=">
+                    <img src="{{ url_for('static', filename='centillion_white.png') }}">
+                    </a>
+                    {#
+                    need a tag line
+                    #}
+                    {% if config['TAGLINE'] %}
+                        <h2><a href="{{ url_for('search')}}?query=&fields=">
+                            {{config['TAGLINE']}}
+                        </a></h2>
+                    {% endif %}
+                </center>
+            </div>
+        </div>
+    
+    </div>
+
     {% block body %}{% endblock %}
 </div>
+
+<a href="https://github.com/dcppc/centillion" class="github-corner" aria-label="View source on Github"><svg width="80" height="80" viewBox="0 0 250 250" style="fill:#151513; color:#fff; position: absolute; top: 0; border: 0; right: 0;" aria-hidden="true"><path d="M0,0 L115,115 L130,115 L142,142 L250,250 L250,0 Z"></path><path d="M128.3,109.0 C113.8,99.7 119.0,89.6 119.0,89.6 C122.0,82.7 120.5,78.6 120.5,78.6 C119.2,72.0 123.4,76.3 123.4,76.3 C127.3,80.9 125.5,87.3 125.5,87.3 C122.9,97.6 130.6,101.9 134.4,103.2" fill="currentColor" style="transform-origin: 130px 106px;" class="octo-arm"></path><path d="M115.0,115.0 C114.9,115.1 118.7,116.5 119.8,115.4 L133.7,101.6 C136.9,99.2 139.9,98.4 142.2,98.6 C133.8,88.0 127.5,74.4 143.8,58.0 C148.5,53.4 154.0,51.2 159.7,51.0 C160.3,49.4 163.2,43.6 171.4,40.1 C171.4,40.1 176.1,42.5 178.8,56.2 C183.1,58.6 187.2,61.8 190.9,65.4 C194.5,69.0 197.7,73.2 200.1,77.6 C213.8,80.2 216.3,84.9 216.3,84.9 C212.7,93.1 206.9,96.0 205.4,96.6 C205.1,102.4 203.0,107.8 198.3,112.5 C181.9,128.9 168.3,122.5 157.7,114.1 C157.9,116.9 156.7,120.9 152.7,124.9 L141.0,136.5 C139.8,137.7 141.6,141.9 141.8,141.8 Z" fill="currentColor" class="octo-body"></path></svg></a><style>.github-corner:hover .octo-arm{animation:octocat-wave 560ms ease-in-out}@keyframes octocat-wave{0%,100%{transform:rotate(0)}20%,60%{transform:rotate(-25deg)}40%,80%{transform:rotate(10deg)}}@media (max-width:500px){.github-corner:hover .octo-arm{animation:none}.github-corner .octo-arm{animation:octocat-wave 560ms ease-in-out}}</style>
+

templates/masterlist.html

@@ -0,0 +1,202 @@
+{% extends "layout.html" %}
+{% block body %}
+
+<hr />
+
+<div class="container">
+
+    <div class="row">
+
+    {#
+     # google drive files panel
+     #}
+    <div class="row">
+        <div class="panel">
+            <div class="panel-group" id="accordionDrive" role="tablist" aria-multiselectable="true">
+                <div class="panel panel-default">
+                    <div class="panel-heading" role="tab" id="googleDrive">
+
+                        <h2 class="masterlist-header">
+                            <a class="collapsed" 
+                                role="button"
+                                onClick="load_gdoc_table()"
+                                data-toggle="collapse" 
+                                data-parent="#accordionDrive"
+                                href="#collapseDrive" 
+                                aria-expanded="true"
+                                aria-controls="collapseDrive">
+                                Google Drive Files <small>indexed by centillion</small>
+                            </a>
+                        </h2>
+
+                    </div>
+                    <div id="collapseDrive" class="panel-collapse collapse" role="tabpanel" aria-labelledby="googleDrive">
+                        <div class="panel-body">
+                            <table class="table table-striped" id="gdocs-master-list">
+
+                            </table>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+
+
+    {#
+     # github issue panel
+     #}
+    <div class="row">
+        <div class="panel">
+            <div class="panel-group" id="accordionIssues" role="tablist" aria-multiselectable="true">
+                <div class="panel panel-default">
+                    <div class="panel-heading" role="tab" id="githubIssues">
+
+                        <h2 class="masterlist-header">
+                            <a class="collapsed" 
+                                role="button"
+                                onClick="load_issue_table()"
+                                data-toggle="collapse" 
+                                data-parent="#accordionIssues"
+                                href="#collapseIssues" 
+                                aria-expanded="true"
+                                aria-controls="collapseIssues">
+                                Github Issues <small>indexed by centillion</small>
+                            </a>
+                        </h2>
+
+                    </div>
+                    <div id="collapseIssues" class="panel-collapse collapse" role="tabpanel" 
+                        aria-labelledby="githubIssues">
+                        <div class="panel-body">
+                            <table class="table table-striped" id="issues-master-list">
+
+                            </table>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+
+
+
+    {#
+     # github file panel
+     #}
+    <div class="row">
+        <div class="panel">
+            <div class="panel-group" id="accordionFiles" role="tablist" aria-multiselectable="true">
+                <div class="panel panel-default">
+                    <div class="panel-heading" role="tab" id="githubFiles">
+
+                        <h2 class="masterlist-header">
+                            <a class="collapsed" 
+                                role="button"
+                                onClick="load_ghfile_table()"
+                                data-toggle="collapse" 
+                                data-parent="#accordionFiles"
+                                href="#collapseFiles" 
+                                aria-expanded="true"
+                                aria-controls="collapseFiles">
+                                Github Files <small>indexed by centillion</small>
+                            </a>
+                        </h2>
+
+                    </div>
+                    <div id="collapseFiles" class="panel-collapse collapse" role="tabpanel" 
+                        aria-labelledby="githubFiles">
+                        <div class="panel-body">
+                            <table class="table table-striped" id="ghfiles-master-list">
+
+                            </table>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+
+    {#
+     # gh markdown file panel
+     #}
+    <div class="row">
+        <div class="panel">
+            <div class="panel-group" id="accordionMarkdown" role="tablist" aria-multiselectable="true">
+                <div class="panel panel-default">
+                    <div class="panel-heading" role="tab" id="markdown">
+
+                        <h2 class="masterlist-header">
+                            <a class="collapsed" 
+                                role="button"
+                                onClick="load_markdown_table()"
+                                data-toggle="collapse" 
+                                data-parent="#accordionMarkdown"
+                                href="#collapseMarkdown" 
+                                aria-expanded="true"
+                                aria-controls="collapseMarkdown">
+                                Github Markdown Files <small>indexed by centillion</small>
+                            </a>
+                        </h2>
+
+                    </div>
+                    <div id="collapseMarkdown" class="panel-collapse collapse" role="tabpanel" 
+                        aria-labelledby="markdown">
+                        <div class="panel-body">
+                            <table class="table table-striped" id="markdown-master-list">
+
+                            </table>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+
+
+    {#
+     # groups.io
+     #}
+    <div class="row">
+        <div class="panel">
+            <div class="panel-group" id="accordionThreads" role="tablist" aria-multiselectable="true">
+                <div class="panel panel-default">
+                    <div class="panel-heading" role="tab" id="threads">
+
+                        <h2 class="masterlist-header">
+                            <a class="collapsed" 
+                                role="button"
+                                onClick="load_emailthreads_table()"
+                                data-toggle="collapse" 
+                                data-parent="#accordionThreads"
+                                href="#collapseThreads" 
+                                aria-expanded="true"
+                                aria-controls="collapseThreads">
+                                Groups.io Email Threads <small>indexed by centillion</small>
+                            </a>
+                        </h2>
+
+                    </div>
+                    <div id="collapseThreads" class="panel-collapse collapse" role="tabpanel" 
+                        aria-labelledby="threads">
+                        <div class="panel-body">
+                            <table class="table table-striped" id="emailthreads-master-list">
+
+                            </table>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+
+
+</div>
+
+{% endblock %}
+

templates/oldsearch.html

@@ -0,0 +1,165 @@
+{% extends "layout.html" %}
+{% block body %}
+
+<div class="container">
+    <div class="row">
+
+        <div class="col-xs-12">
+            <center>
+                <form action="{{ url_for('search') }}" name="search">
+                    <p><input type="text" name="query" value="{{ query }}">
+                    </p>
+                    <p><button id="the-big-one" type="submit" style="font-size: 20px; padding: 10px; padding-left: 50px; padding-right: 50px;" 
+                        value="search" class="btn btn-primary">Search</button>
+                    </p>
+                    <p><a href="{{ url_for('search')}}?query=&fields=">[clear all results]</a>
+                    </p>
+                </form>
+            </center>
+        </div>
+    </div>
+</div>
+
+<div class="container">
+    <div class="row">
+
+        {% if directories %}
+        <div class="col-xs-12 info directories-cloud">
+            <b>File directories:</b> 
+            {% for d in directories %}
+                <a href="{{url_for('search')}}?query={{d|trim}}&fields=filename">{{d|trim}}</a>
+            {% endfor %}
+        </div>
+        {% endif %}
+
+        <ul class="list-group">
+
+            {% if config['SHOW_PARSED_QUERY'] and parsed_query %}
+                <li  class="list-group-item">
+                    <div class="container-fluid">
+                        <div class="row">
+                            <div class="col-xs-12 info">
+                                <b>Parsed query:</b> {{ parsed_query }}
+                            </div>
+                        </div>
+                    </div>
+                </li>
+            {% endif %}
+
+            {% if parsed_query %}
+                <li  class="list-group-item">
+                    <div class="container-fluid">
+                        <div class="row">
+                            <div class="col-xs-12 info">
+                                <b>Found:</b> <span class="badge">{{entries|length}}</span> results 
+                                out of <span class="badge">{{totals["total"]}}</span> total items indexed
+                            </div>
+                        </div>
+                    </div>
+                </li>
+            {% endif %}
+
+            <li  class="list-group-item">
+                    <div class="container-fluid">
+                        <div class="row">
+                            <div class="col-xs-12 info">
+                                <b>Indexing:</b>
+
+                                <span class="badge">{{totals["gdoc"]}}</span>
+                                <a href="/master_list?doctype=gdoc">
+                                Google Drive files
+                                </a>,
+
+                                <span class="badge">{{totals["issue"]}}</span>
+                                <a href="/master_list?doctype=issue">
+                                Github issues
+                                </a>,
+
+                                <span class="badge">{{totals["ghfile"]}}</span>
+                                <a href="/master_list?doctype=ghfile">
+                                Github files
+                                </a>,
+
+                                <span class="badge">{{totals["markdown"]}}</span>
+                                <a href="/master_list?doctype=markdown">
+                                Github Markdown files
+                                </a>,
+
+                                <span class="badge">{{totals["emailthread"]}}</span>
+                                <a href="/master_list?doctype=emailthread">
+                                Groups.io email threads
+                                </a>
+                            </div>
+                        </div>
+                </div>
+            </li>
+
+        </ul>
+    </div>
+</div>
+
+<div class="container">
+    <div class="row">
+        <ul class="list-group">
+
+            {% for e in entries %}
+                <li  class="search-group-item">
+
+                    <div class="url">
+                        {% if e.kind=="gdoc" %}
+                            {% if e.mimetype=="" %}
+                                <b>Google Document:</b>
+                                <a href='{{e.url}}'>{{e.title}}</a>
+                                (Owner: {{e.owner_name}}, {{e.owner_email}})<br />
+                                <b>Document Type</b>: {{e.mimetype}}
+                            {% else %}
+                                <b>Google Drive File:</b>
+                                <a href='{{e.url}}'>{{e.title}}</a><br />
+                                <b>Owner:</b> {{e.owner_name}}, {{e.owner_email}}<br />
+                            {% endif %}
+
+                        {% elif e.kind=="issue" %}
+                            <b>Github Issue:</b>
+                            <a href='{{e.url}}'>{{e.title}}</a>
+                            {% if e.github_user %}<br />
+                            <b>Opened by:</b> <a href='https://github.com/{{e.github_user}}'>@{{e.github_user}}</a>
+                            {% endif %}
+                            <br/>
+                            <b>Repository:</b> <a href='{{e.repo_url}}'>{{e.repo_name}}</a>
+
+                        {% elif e.kind=="markdown" %}
+                            <b>Github Markdown:</b>
+                            <a href='{{e.url}}'>{{e.title}}</a>
+                            <br/>
+                            <b>Repository:</b> <a href='{{e.repo_url}}'>{{e.repo_name}}</a>
+
+                        {% elif e.kind=="emailthread" %}
+                            <b>Groups.io Email Thread:</b>
+                            <a href='{{e.url}}'>{{e.title}}</a>
+                            <br/>
+                            <b>Started By:</b> {{e.owner_name}}
+
+                        {% else %}
+                            <b>Item:</b> (<a href='{{e.url}}'>link</a>)
+
+                        {% endif %}
+                        <br />
+                        Score: {{'%d'  % e.score}}
+                    </div>
+                    <div class="markdown-body">
+                        {% if e.content_highlight %}
+                            {{ e.content_highlight|safe}}
+                        {% else %}
+                        <p>(A preview of this document is not available.)</p>
+                        {% endif %}
+                    </div>
+
+                </li>
+            {% endfor %}
+        </ul>
+
+    </div>
+</div>
+
+{% endblock %}
+

templates/search.html

@@ -1,42 +1,19 @@
 {% extends "layout.html" %}
 {% block body %}
 
-
-<div class="container">
-
-    {#
-    banner image
-    #}
-    <div class="row">
-        <div class="col12sm">
-            <center>
-                <a href="{{ url_for('search')}}?query=&fields=">
-                <img src="{{ url_for('static', filename='centillion_white.png') }}">
-                </a>
-                {#
-                need a tag line
-                #}
-                {% if config['TAGLINE'] %}
-                    <h2><a href="{{ url_for('search')}}?query=&fields=">
-                        {{config['TAGLINE']}}
-                    </a></h2>
-                {% endif %}
-            </center>
-        </div>
-    </div>
-
-</div>
 <div class="container">
     <div class="row">
 
         <div class="col-xs-12">
             <center>
                 <form action="{{ url_for('search') }}" name="search">
-                    <input type="text" name="query" value="{{ query }}"> <br />
-                    <button type="submit" style="font-size: 20px; padding: 10px; padding-left: 50px; padding-right: 50px;" 
+                    <p><input type="text" name="query" value="{{ query }}">
+                    </p>
+                    <p><button id="the-big-one" type="submit" style="font-size: 20px; padding: 10px; padding-left: 50px; padding-right: 50px;" 
                         value="search" class="btn btn-primary">Search</button>
-                    <br />
-                    <a href="{{ url_for('search')}}?query=&fields=">[clear all results]</a>
+                    </p>
+                    <p><a href="{{ url_for('search')}}?query=&fields=">[clear all results]</a>
+                    </p>
                 </form>
             </center>
         </div>
@@ -86,9 +63,32 @@
                     <div class="container-fluid">
                         <div class="row">
                             <div class="col-xs-12 info">
-                                <b>Indexing:</b> <span class="badge">{{totals["documents"]}}</span> Google Documents,
-                                <span class="badge">{{totals["issues"]}}</span> Github issues,
-                                <span class="badge">{{totals["markdown"]}}</span> markdown files.
+                                <b>Indexing:</b>
+
+                                <span class="badge">{{totals["gdoc"]}}</span>
+                                <a href="/master_list?doctype=gdoc">
+                                Google Drive files
+                                </a>,
+
+                                <span class="badge">{{totals["issue"]}}</span>
+                                <a href="/master_list?doctype=issue">
+                                Github issues
+                                </a>,
+
+                                <span class="badge">{{totals["ghfile"]}}</span>
+                                <a href="/master_list?doctype=ghfile">
+                                Github files
+                                </a>,
+
+                                <span class="badge">{{totals["markdown"]}}</span>
+                                <a href="/master_list?doctype=markdown">
+                                Github Markdown files
+                                </a>,
+
+                                <span class="badge">{{totals["emailthread"]}}</span>
+                                <a href="/master_list?doctype=emailthread">
+                                Groups.io email threads
+                                </a>
                             </div>
                         </div>
                 </div>
@@ -98,76 +98,146 @@
     </div>
 </div>
 
+{% if parsed_query %}
 <div class="container">
     <div class="row">
-        <ul class="list-group">
+        <table id="search-results" class="table">
+            <thead id="search-results-header">
+                <tr id="search-results-header-row">
+                    <td id="search-results-score-col">
+                        Score
+                    </td>
+                    <td id="search-results-type-col">
+                        Type
+                    </td>
+                    <td id="search-results-details-col">
+                        Result Details
+                    </td>
+                </tr>
+            </thead>
+            <tbody>
 
-            {% for e in entries %}
-                <li  class="search-group-item">
+                {% for e in entries %}
+                    <tr>
+                        <td>
+                            {{'%d'  % e.score}}
+                        </td>
+                        <td>
+                            {% if e.kind=="gdoc" %}
+                                {% if e.mimetype=="document" %}
+                                    <p><small>Drive Document</small</p>
+                                    <!--
+                                    <i class="fa fa-google fa-2x"></i>
+                                    <i class="fa fa-file-text fa-2x"></i>
+                                    -->
+                                {% else %}
+                                    <p><small>Drive File</small</p>
+                                    <!--
+                                    <i class="fa fa-google fa-2x"></i>
+                                    <i class="fa fa-file-o fa-2x"></i>
+                                    -->
+                                {% endif %}
 
-                    <div class="url">
-                        {% if e.kind=="gdoc" %}
-                            <b>Google Drive File:</b>
-                            <a href='{{e.url}}'>{{e.title}}</a>
-                            (Owner: {{e.owner_name}}, {{e.owner_email}})
+                            {% elif e.kind=="issue" %}
+                                <p><small>Issue</small</p>
+                                <!--
+                                <i class="fa fa-github fa-2x"></i>
+                                <i class="fa fa-question fa-2x"></i>
+                                -->
+
+                            {% elif e.kind=="ghfile" %}
+                                <p><small>Github File</small</p>
+                                <!--
+                                <i class="fa fa-github fa-2x"></i>
+                                <i class="fa fa-file-o fa-2x"></i>
+                                -->
+
+                            {% elif e.kind=="markdown" %}
+                                <p><small>Github Markdown</small</p>
+                                <!--
+                                <i class="fa fa-github fa-2x"></i>
+                                <i class="fa fa-file-text-o fa-2x"></i>
+                                -->
+
+                            {% elif e.kind=="emailthread" %}
+                                <p><small>Email Thread</small</p>
+                                <!--
+                                <i class="fa fa-envelope-o fa-2x"></i>
+                                -->
+
+                            {% else %}
+                                <p><small>Unknown</small</p>
 
-                        {% elif e.kind=="issue" %}
-                            <b>Issue:</b>
-                            <a href='{{e.url}}'>{{e.title}}</a>
-                            {% if e.github_user %}
-                            opened by <a href='https://github.com/{{e.github_user}}'>@{{e.github_user}}</a>
                             {% endif %}
-                            <br/>
-                            <b>Repository:</b> <a href='{{e.repo_url}}'>{{e.repo_name}}</a>
+                        </td>
+                        <td>
+                            <ul class="list-group">
+                                <li  class="search-group-item">
+                                    <div class="url">
+                                        {% if e.kind=="gdoc" %}
+                                            {% if e.mimetype=="" %}
+                                                <b>Google Document:</b>
+                                                <a href='{{e.url}}'>{{e.title}}</a>
+                                                (Owner: {{e.owner_name}}, {{e.owner_email}})<br />
+                                                <b>Document Type</b>: {{e.mimetype}}
+                                            {% else %}
+                                                <b>Google Drive File:</b>
+                                                <a href='{{e.url}}'>{{e.title}}</a><br />
+                                                <b>Owner:</b> {{e.owner_name}}, {{e.owner_email}}<br />
+                                            {% endif %}
 
-                        {% elif e.kind=="markdown" %}
-                            <b>Markdown:</b>
-                            <a href='{{e.url}}'>{{e.title}}</a>
-                            <br/>
-                            <b>Repository:</b> <a href='{{e.repo_url}}'>{{e.repo_name}}</a>
+                                        {% elif e.kind=="issue" %}
+                                            <b>Github Issue:</b>
+                                            <a href='{{e.url}}'>{{e.title}}</a>
+                                            {% if e.github_user %}<br />
+                                            <b>Opened by:</b> <a href='https://github.com/{{e.github_user}}'>@{{e.github_user}}</a>
+                                            {% endif %}
+                                            <br/>
+                                            <b>Repository:</b> <a href='{{e.repo_url}}'>{{e.repo_name}}</a>
 
-                        {% else %}
-                            <b>Item:</b> (<a href='{{e.url}}'>link</a>)
+                                        {% elif e.kind=="ghfile" %}
+                                            <b>Github File:</b>
+                                            <a href='{{e.url}}'>{{e.title}}</a>
+                                            <br/>
+                                            <b>Repository:</b> <a href='{{e.repo_url}}'>{{e.repo_name}}</a>
 
-                        {% endif %}
-                        <br />
-                        score: {{'%d'  % e.score}}
-                    </div>
-                    <div class="markdown-body">{{ e.content_highlight|safe}}</div>
+                                        {% elif e.kind=="markdown" %}
+                                            <b>Github Markdown:</b>
+                                            <a href='{{e.url}}'>{{e.title}}</a>
+                                            <br/>
+                                            <b>Repository:</b> <a href='{{e.repo_url}}'>{{e.repo_name}}</a>
 
-                </li>
-            {% endfor %}
-        </ul>
-
-    </div>
-</div>
-
-<div class="container">
-    <div class="row">
-        <ul class="list-group">
-
-            {% if config['FOOTER_REPO_NAME'] %}
-                {% if config['FOOTER_REPO_ORG'] %}
-
-                    <li  class="list-group-item">
-                        <div class="container-fluid">
-                            <div class="row">
-                                <div class="col-xs-12 info">
-                                    More information about {{config['FOOTER_REPO_NAME']}} can be found
-                                    in the <a href="https://github.com/{{config['FOOTER_REPO_ORG']}}/{{config['FOOTER_REPO_NAME']}}">{{config['FOOTER_REPO_ORG']}}/{{config['FOOTER_REPO_NAME']}}</a>
-                                    repository on Github.
-                                </div>
-                            </div>
-                        </div>
-                    </li>
-
-                {% endif %}
-            {% endif %}
-
-        </ul>
+                                        {% elif e.kind=="emailthread" %}
+                                            <b>Groups.io Email Thread:</b>
+                                            <a href='{{e.url}}'>{{e.title}}</a>
+                                            <br/>
+                                            <b>Started By:</b> {{e.owner_name}}
+
+                                        {% else %}
+                                            <b>Item:</b> (<a href='{{e.url}}'>link</a>)
+
+                                        {% endif %}
+                                    </div>
+                                    <div class="markdown-body">
+                                        {% if e.content_highlight %}
+                                            {{ e.content_highlight|safe}}
+                                        {% else %}
+                                        <p>(A preview of this document is not available.)</p>
+                                        {% endif %}
+                                    </div>
+                                </li>
+                            </ul>
+                        </td>
+                    </tr>
+
+                {% endfor %}
+
+            </tbody>
+        </table>
 
     </div>
 </div>
+{% endif %}
 
 {% endblock %}