Compare commits
1 Commits
testing
...
releases/v
| Author | SHA1 | Date | |
|---|---|---|---|
| 2099c60c7a |
2
.gitignore
vendored
2
.gitignore
vendored
@@ -1,4 +1,4 @@
|
|||||||
feedback_database.json
|
config_centillion.py
|
||||||
config_flask.py
|
config_flask.py
|
||||||
vp
|
vp
|
||||||
credentials.json
|
credentials.json
|
||||||
|
|||||||
@@ -12,13 +12,8 @@ one centillion is 3.03 log-times better than a googol.
|
|||||||
## What Is It
|
## What Is It
|
||||||
|
|
||||||
Centillion (https://github.com/dcppc/centillion) is a search engine that can index
|
Centillion (https://github.com/dcppc/centillion) is a search engine that can index
|
||||||
different kinds of document collections: Google Documents (.docx files), Google Drive files,
|
three kinds of collections: Google Documents (.docx files), Github issues, and Markdown files in
|
||||||
Github issues, Github files, Github Markdown files, and Groups.io email threads.
|
Github repos.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## What Is It
|
|
||||||
|
|
||||||
We define the types of documents the centillion should index,
|
We define the types of documents the centillion should index,
|
||||||
what info and how. The centillion then builds and
|
what info and how. The centillion then builds and
|
||||||
|
|||||||
56
Todo.md
56
Todo.md
@@ -1,29 +1,47 @@
|
|||||||
# todo
|
# todo
|
||||||
|
|
||||||
ux improvements:
|
Main task:
|
||||||
- feedback tools
|
- hashing and caching
|
||||||
- integrating master list into single list
|
- <s>first, working out the logic of how we group items into sets
|
||||||
- providing advanced search interfce
|
- needs to be deleted
|
||||||
|
- needs to be updated
|
||||||
|
- needs to be added
|
||||||
|
- for docs, issues, and comments</s>
|
||||||
|
- second, when we add or update an item, need to:
|
||||||
|
- go through the motions, download file, extract text
|
||||||
|
- check for existing indexed doc with that id
|
||||||
|
- check if existing indexed doc has same hash
|
||||||
|
- if so, skip
|
||||||
|
- otherwise, delete and re-index
|
||||||
|
|
||||||
big picture improvements:
|
Other bugs:
|
||||||
- hypothesis API
|
- Some github issues have no title (?)
|
||||||
- folksonomy tagging with hypothesis
|
- <s>Need to combine issues with comments</s>
|
||||||
- tags, expanded schema
|
- Not able to index markdown files _in a repo_
|
||||||
|
- (Longer term) update main index vs update diff index
|
||||||
|
|
||||||
|
Needs:
|
||||||
|
- <s>control panel</s>
|
||||||
|
|
||||||
|
Thursday product:
|
||||||
|
- Everything re-indexed nightly
|
||||||
|
- Search engine built on all documents in Google Drive, all issues, markdown files
|
||||||
|
- Using pandoc to extract Google Drive document contents
|
||||||
|
- BRIEF quickstart documentation
|
||||||
|
|
||||||
|
Future:
|
||||||
|
- Future plans to improve - plugins, improving matching
|
||||||
|
- Subdomain plans
|
||||||
|
- Folksonomy tagging and integration plans
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
feedback form: where we are at
|
config options for plugins
|
||||||
- feedback button
|
conditional blocks with import github inside
|
||||||
- button triggers modal form
|
complicated tho - better to have components split off
|
||||||
- modal has emojis for feedback, text box, buttons
|
|
||||||
- clicking emojis changes color, to select
|
|
||||||
- clicking submit with filled out form submits to an endpoint
|
|
||||||
- clicking submit also closes form, but only if submit successful
|
|
||||||
|
|
||||||
feedback form: what we need to do
|
|
||||||
- fix alerts - thank you for your feedback doesn't show up until a refresh
|
|
||||||
- probably an easy ajax fix
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -3,7 +3,6 @@ import subprocess
|
|||||||
|
|
||||||
import codecs
|
import codecs
|
||||||
import os, json
|
import os, json
|
||||||
from datetime import datetime
|
|
||||||
|
|
||||||
from werkzeug.contrib.fixers import ProxyFix
|
from werkzeug.contrib.fixers import ProxyFix
|
||||||
from flask import Flask, request, redirect, url_for, render_template, flash, jsonify
|
from flask import Flask, request, redirect, url_for, render_template, flash, jsonify
|
||||||
@@ -263,57 +262,13 @@ def list_docs(doctype):
|
|||||||
all_orgs = resp.json()
|
all_orgs = resp.json()
|
||||||
for org in all_orgs:
|
for org in all_orgs:
|
||||||
if org['login']=='dcppc':
|
if org['login']=='dcppc':
|
||||||
|
copper_team_id = '2700235'
|
||||||
|
mresp = github.get('/teams/%s/members/%s'%(copper_team_id,username))
|
||||||
|
if mresp.status_code==204:
|
||||||
# Business as usual
|
# Business as usual
|
||||||
search = Search(app.config["INDEX_DIR"])
|
search = Search(app.config["INDEX_DIR"])
|
||||||
return jsonify(search.get_list(doctype))
|
return jsonify(search.get_list(doctype))
|
||||||
|
|
||||||
# nope
|
|
||||||
return render_template('403.html')
|
|
||||||
|
|
||||||
|
|
||||||
@app.route('/feedback', methods=['POST'])
|
|
||||||
def parse_request():
|
|
||||||
|
|
||||||
if not github.authorized:
|
|
||||||
return redirect(url_for("github.login"))
|
|
||||||
username = github.get("/user").json()['login']
|
|
||||||
resp = github.get("/user/orgs")
|
|
||||||
if resp.ok:
|
|
||||||
all_orgs = resp.json()
|
|
||||||
for org in all_orgs:
|
|
||||||
if org['login']=='dcppc':
|
|
||||||
|
|
||||||
|
|
||||||
try:
|
|
||||||
# Business as usual
|
|
||||||
data = request.form.to_dict();
|
|
||||||
data['github_login'] = username
|
|
||||||
data['timestamp'] = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
|
||||||
|
|
||||||
feedback_database = 'feedback_database.json'
|
|
||||||
if not os.path.isfile(feedback_database):
|
|
||||||
with open(feedback_database,'w') as f:
|
|
||||||
json_data = [data]
|
|
||||||
json.dump(json_data, f, indent=4)
|
|
||||||
|
|
||||||
else:
|
|
||||||
json_data = []
|
|
||||||
with open(feedback_database,'r') as f:
|
|
||||||
json_data = json.load(f)
|
|
||||||
|
|
||||||
json_data.append(data)
|
|
||||||
|
|
||||||
with open(feedback_database,'w') as f:
|
|
||||||
json.dump(json_data, f, indent=4)
|
|
||||||
|
|
||||||
## Should be done with Javascript
|
|
||||||
#flash("Thank you for your feedback!")
|
|
||||||
return jsonify({'status':'ok','message':'Thank you for your feedback!'})
|
|
||||||
except:
|
|
||||||
return jsonify({'status':'error','message':'An error was encountered while submitting your feedback. Try submitting an issue in the <a href="https://github.com/dcppc/centillion/issues/new">dcppc/centillion</a> repository.'})
|
|
||||||
|
|
||||||
|
|
||||||
# nope
|
|
||||||
return render_template('403.html')
|
return render_template('403.html')
|
||||||
|
|
||||||
@app.errorhandler(404)
|
@app.errorhandler(404)
|
||||||
@@ -342,10 +297,5 @@ def store_search(query, fields):
|
|||||||
if __name__ == '__main__':
|
if __name__ == '__main__':
|
||||||
# if running local instance, set to true
|
# if running local instance, set to true
|
||||||
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = 'true'
|
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = 'true'
|
||||||
port = os.environ.get('CENTILLION_PORT','')
|
app.run(host="0.0.0.0",port=5000)
|
||||||
if port=='':
|
|
||||||
port = 5000
|
|
||||||
else:
|
|
||||||
port = int(port)
|
|
||||||
app.run(host="0.0.0.0",port=port)
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,28 +0,0 @@
|
|||||||
config = {
|
|
||||||
"repositories" : [
|
|
||||||
"dcppc/project-management",
|
|
||||||
"dcppc/nih-demo-meetings",
|
|
||||||
"dcppc/internal",
|
|
||||||
"dcppc/organize",
|
|
||||||
"dcppc/dcppc-bot",
|
|
||||||
"dcppc/full-stacks",
|
|
||||||
"dcppc/design-guidelines-discuss",
|
|
||||||
"dcppc/dcppc-deliverables",
|
|
||||||
"dcppc/dcppc-milestones",
|
|
||||||
"dcppc/crosscut-metadata",
|
|
||||||
"dcppc/lucky-penny",
|
|
||||||
"dcppc/dcppc-workshops",
|
|
||||||
"dcppc/metadata-matrix",
|
|
||||||
"dcppc/data-stewards",
|
|
||||||
"dcppc/dcppc-phase1-demos",
|
|
||||||
"dcppc/apis",
|
|
||||||
"dcppc/2018-june-workshop",
|
|
||||||
"dcppc/2018-july-workshop",
|
|
||||||
"dcppc/2018-august-workshop",
|
|
||||||
"dcppc/2018-september-workshop",
|
|
||||||
"dcppc/design-guidelines",
|
|
||||||
"dcppc/2018-may-workshop",
|
|
||||||
"dcppc/centillion"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
|
|
||||||
@@ -1,181 +0,0 @@
|
|||||||
# Centillion Quality Engineering Plan
|
|
||||||
|
|
||||||
Table of Contents
|
|
||||||
-------
|
|
||||||
|
|
||||||
* [Centillion Quality Engineering Plan](#centillion-quality-engineering-plan)
|
|
||||||
* [Summary](#summary)
|
|
||||||
* [Tracking Bugs and Issues](#tracking-bugs-and-issues)
|
|
||||||
* [Branches, Versioning, and Git Workflow](#branches-versioning-and-git-workflow)
|
|
||||||
* [Communication and Mailing Lists](#communication-and-mailing-lists)
|
|
||||||
* [Checklists](#checklists)
|
|
||||||
* [Documentation](#documentation)
|
|
||||||
* [Configuration Management Tools](#configuration-management-tools)
|
|
||||||
* [Tests](#tests)
|
|
||||||
* [Code Reviews](#code-reviews)
|
|
||||||
* [Formal Release Process](#formal-release-process)
|
|
||||||
* [Continual Process Improvement](#continual-process-improvement)
|
|
||||||
|
|
||||||
Summary
|
|
||||||
-------
|
|
||||||
|
|
||||||
This document contains a quality engineering plan for centillion, the
|
|
||||||
Data Commons search engine.
|
|
||||||
|
|
||||||
Tracking Bugs and Issues
|
|
||||||
------------------------
|
|
||||||
|
|
||||||
We utilize the [issues
|
|
||||||
section](https://github.com/dcppc/centillion/issues) of the centillion
|
|
||||||
repository to keep track of bugs and feature requests.
|
|
||||||
|
|
||||||
Branches, Versioning, and Git Workflow
|
|
||||||
--------------------------------------
|
|
||||||
|
|
||||||
All code is kept under version control in the
|
|
||||||
[dcppc/centillion](https://github.com/dcppc/centillion) Github
|
|
||||||
repository.
|
|
||||||
|
|
||||||
**Primary Git Branches:**
|
|
||||||
|
|
||||||
We utillize a git branch pattern that has two primary branches: a
|
|
||||||
development branch and a stable branch.
|
|
||||||
|
|
||||||
- The primary **development branch** is `dcppc` and is actively
|
|
||||||
developed and deployed to <https://betasearch.nihdatacommons.us>.
|
|
||||||
|
|
||||||
- The primary **stable branch** is `releases/v1` and is stable and
|
|
||||||
deployed to <https://search.nihdatacommons.us>.
|
|
||||||
|
|
||||||
All tagged versions of Centillion exist on the stable branch. Only
|
|
||||||
tagged versions of centillion are run on
|
|
||||||
<https://search.nihdatacommons.us>.
|
|
||||||
|
|
||||||
**Other Branches:**
|
|
||||||
|
|
||||||
Features are developed by creating a new branch from `dcppc`, working on
|
|
||||||
the feature, and opening a pull request. When the pull request is
|
|
||||||
approved, it can be merged into the `dcppc` branch.
|
|
||||||
|
|
||||||
When features have accumulated and a new version is ready, a new
|
|
||||||
pre-release branch will be made to prepare for a new release. When the
|
|
||||||
pre-release branch is ready, it is merged into the stable branch in a
|
|
||||||
single merge commit and a new version of centillion is tagged. The new
|
|
||||||
version is deployed on <https://search.nihdatacommons.us>.
|
|
||||||
|
|
||||||
Commits to fix bugs (hotfixes) may need to be applied to both the stable
|
|
||||||
and development branches. In this case, a hotfix branch should be
|
|
||||||
created from the head commit of the stable branch, and the appropriate
|
|
||||||
changes should be made on the branch. A pull request should be opened to
|
|
||||||
merge the hotfix into the release branch. A second pull request should
|
|
||||||
be opened to merge the hotfix into the development branch. Once the
|
|
||||||
hotfix is merged into the stable branch, a new version should be tagged.
|
|
||||||
|
|
||||||
Communication and Mailing Lists
|
|
||||||
-------------------------------
|
|
||||||
|
|
||||||
- No mailing list currently exists for centillion.
|
|
||||||
|
|
||||||
- Github issues are the primary form of communication about
|
|
||||||
development of centillion. This is the best method for communicating
|
|
||||||
bug reports or detailed information.
|
|
||||||
|
|
||||||
- The Send Feedback button on the centillion page is the primary way
|
|
||||||
of getting quick feedback from users about the search engine.
|
|
||||||
|
|
||||||
- The [\#centillion](https://nih-dcppc.slack.com/messages/CCD64QD6G)
|
|
||||||
Slack channel in the DCPPC slack workspace is the best place for
|
|
||||||
conversations about centillion (providing feedback, answering quick
|
|
||||||
questions, etc.)
|
|
||||||
|
|
||||||
Checklists
|
|
||||||
----------
|
|
||||||
|
|
||||||
We plan to utilize the Wiki feature of the Github repository to develop
|
|
||||||
checlists:
|
|
||||||
|
|
||||||
- Checklist for releases
|
|
||||||
- Checklist for deployment of https://search.nihdatacommons.us nginx
|
|
||||||
etc.
|
|
||||||
|
|
||||||
Documentation
|
|
||||||
-------------
|
|
||||||
|
|
||||||
The documentation is a pile of markdown documents, turned into a static
|
|
||||||
site using mkdocs.
|
|
||||||
|
|
||||||
Configuration Management Tools
|
|
||||||
------------------------------
|
|
||||||
|
|
||||||
We do not currently utilize any configuration management software,
|
|
||||||
because centillion is not packaged as an importable Python module.
|
|
||||||
|
|
||||||
Packaging centillion is a future goal that is closely related to the
|
|
||||||
need to improve and modularize the internal search schema/document type
|
|
||||||
abstraction. These improvements would allow the types of collections
|
|
||||||
being indexed to be separated from "core centillion", and core
|
|
||||||
centillion would be packaged.
|
|
||||||
|
|
||||||
Tests
|
|
||||||
-----
|
|
||||||
|
|
||||||
See (ref) for a full test plan with more detail.
|
|
||||||
|
|
||||||
Summary of test plan:
|
|
||||||
|
|
||||||
- Implement tests for the four major pages/components
|
|
||||||
- Login/authentication
|
|
||||||
- Search
|
|
||||||
- Master List
|
|
||||||
- Control Panel
|
|
||||||
- Test authentication with two bot accounts (yammasnake and florence
|
|
||||||
python)
|
|
||||||
|
|
||||||
- Separate frontend and backend tests
|
|
||||||
|
|
||||||
- Add a test flag in the flask config file to change the backend
|
|
||||||
behavior of the server
|
|
||||||
|
|
||||||
Code Reviews
|
|
||||||
------------
|
|
||||||
|
|
||||||
CI tests will be implemented for all pull requests.
|
|
||||||
|
|
||||||
Pull requests to the **stable branch** have the following checks in
|
|
||||||
place:
|
|
||||||
|
|
||||||
- PRs to the stable branch require at least 1 PR review
|
|
||||||
- PRs to the stable branch must pass CI tests
|
|
||||||
|
|
||||||
Pull requests to the **development branch** have the following checks in
|
|
||||||
place:
|
|
||||||
|
|
||||||
- PRs to the development branch must pass CI tests
|
|
||||||
|
|
||||||
Formal Release Process
|
|
||||||
----------------------
|
|
||||||
|
|
||||||
In order to ensure a stable, consistent product, we utilize the
|
|
||||||
branching pattern described above to implement new features in the
|
|
||||||
development branch and test them out on
|
|
||||||
<https://betasearch.nihdatacommons.us>.
|
|
||||||
|
|
||||||
Once features and bug fixes have been tested and reviewed internally,
|
|
||||||
they are ready to be deployed. A new pre-release branch is created from
|
|
||||||
the development branch. The pre-release branch has a feature freeze in
|
|
||||||
place. Changes are made to the pre-release branch to prepare it for the
|
|
||||||
next major version release.
|
|
||||||
|
|
||||||
When the pre-release branch is finished, it is merged into the stable
|
|
||||||
branch. The head commit of the stable version is tagged with the lastest
|
|
||||||
release number.
|
|
||||||
|
|
||||||
Finally, the new version is deployed on
|
|
||||||
<https://search.nihdatacommons.us>.
|
|
||||||
|
|
||||||
Continual Process Improvement
|
|
||||||
-----------------------------
|
|
||||||
|
|
||||||
We will utilize the centillion wiki on Github to keep track of repeated
|
|
||||||
processes and opportunities for improvement. Feedback and ideas for
|
|
||||||
process improvement can also be submitted via Github issues.
|
|
||||||
@@ -1,133 +0,0 @@
|
|||||||
// submitting form with modal:
|
|
||||||
// https://stackoverflow.com/a/29068742
|
|
||||||
//
|
|
||||||
// closing a bootstrap modal with submit button:
|
|
||||||
// https://stackoverflow.com/a/33478107
|
|
||||||
//
|
|
||||||
// flask post data as json:
|
|
||||||
// https://stackoverflow.com/a/16664376
|
|
||||||
|
|
||||||
/* this function is called when the user submits
|
|
||||||
* the feedback form. it submits a post request
|
|
||||||
* to the flask server, which squirrels away the
|
|
||||||
* feedback in a file.
|
|
||||||
*/
|
|
||||||
function submit_feedback() {
|
|
||||||
// this function is called when submit button clicked
|
|
||||||
// algorithm:
|
|
||||||
// - check if text box has content
|
|
||||||
// - check if happy/sad filled out
|
|
||||||
|
|
||||||
var smile_active = $('#modal-feedback-smile-div').hasClass('smile-active');
|
|
||||||
var frown_active = $('#modal-feedback-frown-div').hasClass('frown-active');
|
|
||||||
if( !( smile_active || frown_active ) ) {
|
|
||||||
alert('Please pick the smile or the frown.')
|
|
||||||
} else if( $('#modal-feedback-textarea').val()=='' ) {
|
|
||||||
alert('Please provide us with some feedback.')
|
|
||||||
} else {
|
|
||||||
var user_sentiment = '';
|
|
||||||
if(smile_active) {
|
|
||||||
user_sentiment = 'smile';
|
|
||||||
} else {
|
|
||||||
user_sentiment = 'frown';
|
|
||||||
}
|
|
||||||
var escaped_text = $('#modal-feedback-textarea').val();
|
|
||||||
|
|
||||||
// prepare form data
|
|
||||||
var data = {
|
|
||||||
sentiment : user_sentiment,
|
|
||||||
content : escaped_text
|
|
||||||
};
|
|
||||||
// post the form. the callback function resets the form
|
|
||||||
$.post("/feedback",
|
|
||||||
data,
|
|
||||||
function(response) {
|
|
||||||
$('#myModal').modal('hide');
|
|
||||||
$('#myModalForm')[0].reset();
|
|
||||||
add_alert(response);
|
|
||||||
frown_unclick();
|
|
||||||
smile_unclick();
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
function add_alert(response) {
|
|
||||||
str = ""
|
|
||||||
str += '<div id="feedback-messages-container" class="container">';
|
|
||||||
|
|
||||||
if (response['status']=='ok') {
|
|
||||||
// if status is ok, use alert-success
|
|
||||||
str += ' <div id="feedback-messages-alert" class="alert alert-success alert-dismissible fade in">';
|
|
||||||
} else {
|
|
||||||
// otherwise use alert-danger
|
|
||||||
str += ' <div id="feedback-messages-alert" class="alert alert-danger alert-dismissible fade in">';
|
|
||||||
}
|
|
||||||
|
|
||||||
str += ' <a href="#" class="close" data-dismiss="alert" aria-label="close">×</a>';
|
|
||||||
str += ' <div id="feedback-messages-contianer" class="container-fluid">';
|
|
||||||
str += ' <div id="feedback-messages-div" class="co-xs-12">';
|
|
||||||
str += ' <p>'
|
|
||||||
str += response['message'];
|
|
||||||
str += ' </p>';
|
|
||||||
str += ' </div>';
|
|
||||||
str += ' </div>';
|
|
||||||
str += '</div>';
|
|
||||||
$('div#messages').append(str);
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
/* for those particularly wordy users... limit feedback to 1000 chars */
|
|
||||||
function cool_it() {
|
|
||||||
if($('#modal-feedback-textarea').val().length > 1100 ){
|
|
||||||
$('#modal-too-long').show();
|
|
||||||
} else {
|
|
||||||
$('#modal-too-long').hide();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/* smiley functions */
|
|
||||||
function smile_click() {
|
|
||||||
$('#modal-feedback-smile-div').addClass('smile-active');
|
|
||||||
$('#modal-feedback-smile-icon').addClass('smile-active');
|
|
||||||
}
|
|
||||||
function frown_click() {
|
|
||||||
$('#modal-feedback-frown-div').addClass('frown-active');
|
|
||||||
$('#modal-feedback-frown-icon').addClass('frown-active');
|
|
||||||
}
|
|
||||||
function smile_unclick() {
|
|
||||||
$('#modal-feedback-smile-div').removeClass('smile-active');
|
|
||||||
$('#modal-feedback-smile-icon').removeClass('smile-active');
|
|
||||||
}
|
|
||||||
function frown_unclick() {
|
|
||||||
$('#modal-feedback-frown-div').removeClass('frown-active');
|
|
||||||
$('#modal-feedback-frown-icon').removeClass('frown-active');
|
|
||||||
}
|
|
||||||
|
|
||||||
function smile() {
|
|
||||||
frown_unclick();
|
|
||||||
smile_click();
|
|
||||||
}
|
|
||||||
function frown() {
|
|
||||||
smile_unclick();
|
|
||||||
frown_click();
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
/* for those particularly wordy users... limit feedback to 1100 chars */
|
|
||||||
// how to check n characters in a textarea
|
|
||||||
// https://stackoverflow.com/a/19934613
|
|
||||||
/*
|
|
||||||
$(document).ready(function() {
|
|
||||||
|
|
||||||
$('#modal-feedback-textarea').on('change',function(event) {
|
|
||||||
if($('#modal-feedback-textarea').val().length > 1100 ){
|
|
||||||
$('#modal-too-long').show();
|
|
||||||
} else {
|
|
||||||
$('#modal-too-long').hide();
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
}
|
|
||||||
*/
|
|
||||||
|
|
||||||
@@ -1,61 +1,3 @@
|
|||||||
#modal-too-long {
|
|
||||||
visibility: hidden;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* feedback smileys */
|
|
||||||
#modal-feedback-smile-icon,
|
|
||||||
#modal-feedback-frown-icon {
|
|
||||||
padding-left: 100px;
|
|
||||||
padding-right: 100px;
|
|
||||||
padding-top: 20px;
|
|
||||||
padding-bottom: 20px;
|
|
||||||
}
|
|
||||||
|
|
||||||
div.smile-active {
|
|
||||||
background-color: #2b2;
|
|
||||||
}
|
|
||||||
i.smile-active {
|
|
||||||
color: #fff;
|
|
||||||
}
|
|
||||||
div.frown-active {
|
|
||||||
background-color: #b22;
|
|
||||||
}
|
|
||||||
i.frown-active {
|
|
||||||
color: #fff;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* feedback text area */
|
|
||||||
#modal-feedback-textarea {
|
|
||||||
width: 100%;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* feedback buttons */
|
|
||||||
button.close {
|
|
||||||
font-size: 35px;
|
|
||||||
}
|
|
||||||
button#submit-feedback-btn {
|
|
||||||
width: 250px;
|
|
||||||
}
|
|
||||||
|
|
||||||
button#feedback:hover {
|
|
||||||
opacity: 1.0;
|
|
||||||
filter: alpha(opacity=100); /* For IE8 and earlier */
|
|
||||||
}
|
|
||||||
|
|
||||||
button#feedback {
|
|
||||||
opacity: 0.5;
|
|
||||||
filter: alpha(opacity=50); /* For IE8 and earlier */
|
|
||||||
|
|
||||||
width: 180px;
|
|
||||||
height: 50px;
|
|
||||||
position: fixed;
|
|
||||||
z-index: 999;
|
|
||||||
right: 120px;
|
|
||||||
bottom: 10px;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
/* search results table */
|
|
||||||
td#search-results-score-col,
|
td#search-results-score-col,
|
||||||
td#search-results-type-col {
|
td#search-results-type-col {
|
||||||
width: 100px;
|
width: 100px;
|
||||||
|
|||||||
@@ -1,5 +1,4 @@
|
|||||||
{% extends "layout.html" %}
|
{% extends "layout.html" %}
|
||||||
{% set active_page = "403" %}
|
|
||||||
{% block body %}
|
{% block body %}
|
||||||
|
|
||||||
<div class="container">
|
<div class="container">
|
||||||
|
|||||||
@@ -1,5 +1,4 @@
|
|||||||
{% extends "layout.html" %}
|
{% extends "layout.html" %}
|
||||||
{% set active_page = "404" %}
|
|
||||||
{% block body %}
|
{% block body %}
|
||||||
|
|
||||||
<div class="container">
|
<div class="container">
|
||||||
|
|||||||
@@ -1,25 +0,0 @@
|
|||||||
<div class="container" id="banner-container">
|
|
||||||
|
|
||||||
{#
|
|
||||||
banner image
|
|
||||||
#}
|
|
||||||
<div class="row" id="banner-row">
|
|
||||||
<div class="col12sm" id="banner-col">
|
|
||||||
<center>
|
|
||||||
<a id="banner-a" href="{{ url_for('search')}}?query=&fields=">
|
|
||||||
<img id="banner-img" src="{{ url_for('static', filename='centillion_white.png') }}">
|
|
||||||
</a>
|
|
||||||
</center>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
{% if config['TAGLINE'] %}
|
|
||||||
<div class="row" id="tagline-row">
|
|
||||||
<div class="col12sm" id="tagline-col">
|
|
||||||
<center>
|
|
||||||
<h2 id="tagline-tagline"> {{config['TAGLINE']}} </h2>
|
|
||||||
</center>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
</div>
|
|
||||||
@@ -1,5 +1,4 @@
|
|||||||
{% extends "layout.html" %}
|
{% extends "layout.html" %}
|
||||||
{% set active_page = "control_panel" %}
|
|
||||||
{% block body %}
|
{% block body %}
|
||||||
|
|
||||||
<hr />
|
<hr />
|
||||||
|
|||||||
@@ -1,14 +0,0 @@
|
|||||||
<div id="messages">
|
|
||||||
{% with messages = get_flashed_messages() %}
|
|
||||||
{% if messages %}
|
|
||||||
<div class="container" id="flashed-messages-container">
|
|
||||||
<div class="alert alert-success alert-dismissible fade in">
|
|
||||||
<a href="#" class="close" data-dismiss="alert" aria-label="close">×</a>
|
|
||||||
{% for message in messages %}
|
|
||||||
<p class="lead">{{ message }}</p>
|
|
||||||
{% endfor %}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
{% endif %}
|
|
||||||
{% endwith %}
|
|
||||||
</div>
|
|
||||||
@@ -1,5 +1,4 @@
|
|||||||
{% extends "layout.html" %}
|
{% extends "layout.html" %}
|
||||||
{% set active_page = "landing" %}
|
|
||||||
{% block body %}
|
{% block body %}
|
||||||
<div class="container">
|
<div class="container">
|
||||||
<div class="row">
|
<div class="row">
|
||||||
|
|||||||
@@ -10,7 +10,6 @@
|
|||||||
<script src="{{ url_for('static', filename='bootstrap.min.js') }}"></script>
|
<script src="{{ url_for('static', filename='bootstrap.min.js') }}"></script>
|
||||||
<script src="{{ url_for('static', filename='master_list.js') }}"></script>
|
<script src="{{ url_for('static', filename='master_list.js') }}"></script>
|
||||||
<script src="{{ url_for('static', filename='search_list.js') }}"></script>
|
<script src="{{ url_for('static', filename='search_list.js') }}"></script>
|
||||||
<script src="{{ url_for('static', filename='feedback.js') }}"></script>
|
|
||||||
|
|
||||||
|
|
||||||
{# ########## dataTables plugin ############ #}
|
{# ########## dataTables plugin ############ #}
|
||||||
@@ -24,43 +23,52 @@
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
<div id="master-div">
|
{# ########## github fork corner ############ #}
|
||||||
|
|
||||||
{#
|
<div>
|
||||||
flashed messages
|
|
||||||
#}
|
{% with messages = get_flashed_messages() %}
|
||||||
{% include "flashed_messages.html" %}
|
{% if messages %}
|
||||||
|
<div class="container">
|
||||||
|
<div class="alert alert-success alert-dismissible">
|
||||||
|
<a href="#" class="close" data-dismiss="alert" aria-label="close">×</a>
|
||||||
|
<ul class=flashes>
|
||||||
|
{% for message in messages %}
|
||||||
|
<li>{{ message }}</li>
|
||||||
|
{% endfor %}
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
{% endwith %}
|
||||||
|
|
||||||
|
<div class="container">
|
||||||
|
|
||||||
{#
|
{#
|
||||||
banner image
|
banner image
|
||||||
#}
|
#}
|
||||||
{% include "banner.html" %}
|
<div class="row">
|
||||||
|
<div class="col12sm">
|
||||||
|
<center>
|
||||||
|
<a href="{{ url_for('search')}}?query=&fields=">
|
||||||
|
<img src="{{ url_for('static', filename='centillion_white.png') }}">
|
||||||
|
</a>
|
||||||
{#
|
{#
|
||||||
feedback modal
|
need a tag line
|
||||||
#}
|
#}
|
||||||
{% include "modal.html" %}
|
{% if config['TAGLINE'] %}
|
||||||
|
<h2><a href="{{ url_for('search')}}?query=&fields=">
|
||||||
{% block body %}{% endblock %}
|
{{config['TAGLINE']}}
|
||||||
|
</a></h2>
|
||||||
</div>
|
{% endif %}
|
||||||
|
</center>
|
||||||
{% if active_page=="search" or active_page=="master_list" %}
|
</div>
|
||||||
|
|
||||||
{# feedback button #}
|
|
||||||
<button id="feedback" type="button"
|
|
||||||
data-toggle="modal"
|
|
||||||
data-target="#myModal"
|
|
||||||
class="btn btn-lg">Send Feedback</button>
|
|
||||||
|
|
||||||
{# vertical spacing before the bottom, b/c of button #}
|
|
||||||
<div id="footer-whitespace" class="container">
|
|
||||||
<p> </p>
|
|
||||||
<p> </p>
|
|
||||||
<p> </p>
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
{% endif %}
|
</div>
|
||||||
|
|
||||||
<a id="github-corner" href="https://github.com/dcppc/centillion" class="github-corner" aria-label="View source on Github"><svg width="80" height="80" viewBox="0 0 250 250" style="fill:#151513; color:#fff; position: absolute; top: 0; border: 0; right: 0;" aria-hidden="true"><path d="M0,0 L115,115 L130,115 L142,142 L250,250 L250,0 Z"></path><path d="M128.3,109.0 C113.8,99.7 119.0,89.6 119.0,89.6 C122.0,82.7 120.5,78.6 120.5,78.6 C119.2,72.0 123.4,76.3 123.4,76.3 C127.3,80.9 125.5,87.3 125.5,87.3 C122.9,97.6 130.6,101.9 134.4,103.2" fill="currentColor" style="transform-origin: 130px 106px;" class="octo-arm"></path><path d="M115.0,115.0 C114.9,115.1 118.7,116.5 119.8,115.4 L133.7,101.6 C136.9,99.2 139.9,98.4 142.2,98.6 C133.8,88.0 127.5,74.4 143.8,58.0 C148.5,53.4 154.0,51.2 159.7,51.0 C160.3,49.4 163.2,43.6 171.4,40.1 C171.4,40.1 176.1,42.5 178.8,56.2 C183.1,58.6 187.2,61.8 190.9,65.4 C194.5,69.0 197.7,73.2 200.1,77.6 C213.8,80.2 216.3,84.9 216.3,84.9 C212.7,93.1 206.9,96.0 205.4,96.6 C205.1,102.4 203.0,107.8 198.3,112.5 C181.9,128.9 168.3,122.5 157.7,114.1 C157.9,116.9 156.7,120.9 152.7,124.9 L141.0,136.5 C139.8,137.7 141.6,141.9 141.8,141.8 Z" fill="currentColor" class="octo-body"></path></svg></a><style>.github-corner:hover .octo-arm{animation:octocat-wave 560ms ease-in-out}@keyframes octocat-wave{0%,100%{transform:rotate(0)}20%,60%{transform:rotate(-25deg)}40%,80%{transform:rotate(10deg)}}@media (max-width:500px){.github-corner:hover .octo-arm{animation:none}.github-corner .octo-arm{animation:octocat-wave 560ms ease-in-out}}</style>
|
{% block body %}{% endblock %}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<a href="https://github.com/dcppc/centillion" class="github-corner" aria-label="View source on Github"><svg width="80" height="80" viewBox="0 0 250 250" style="fill:#151513; color:#fff; position: absolute; top: 0; border: 0; right: 0;" aria-hidden="true"><path d="M0,0 L115,115 L130,115 L142,142 L250,250 L250,0 Z"></path><path d="M128.3,109.0 C113.8,99.7 119.0,89.6 119.0,89.6 C122.0,82.7 120.5,78.6 120.5,78.6 C119.2,72.0 123.4,76.3 123.4,76.3 C127.3,80.9 125.5,87.3 125.5,87.3 C122.9,97.6 130.6,101.9 134.4,103.2" fill="currentColor" style="transform-origin: 130px 106px;" class="octo-arm"></path><path d="M115.0,115.0 C114.9,115.1 118.7,116.5 119.8,115.4 L133.7,101.6 C136.9,99.2 139.9,98.4 142.2,98.6 C133.8,88.0 127.5,74.4 143.8,58.0 C148.5,53.4 154.0,51.2 159.7,51.0 C160.3,49.4 163.2,43.6 171.4,40.1 C171.4,40.1 176.1,42.5 178.8,56.2 C183.1,58.6 187.2,61.8 190.9,65.4 C194.5,69.0 197.7,73.2 200.1,77.6 C213.8,80.2 216.3,84.9 216.3,84.9 C212.7,93.1 206.9,96.0 205.4,96.6 C205.1,102.4 203.0,107.8 198.3,112.5 C181.9,128.9 168.3,122.5 157.7,114.1 C157.9,116.9 156.7,120.9 152.7,124.9 L141.0,136.5 C139.8,137.7 141.6,141.9 141.8,141.8 Z" fill="currentColor" class="octo-body"></path></svg></a><style>.github-corner:hover .octo-arm{animation:octocat-wave 560ms ease-in-out}@keyframes octocat-wave{0%,100%{transform:rotate(0)}20%,60%{transform:rotate(-25deg)}40%,80%{transform:rotate(10deg)}}@media (max-width:500px){.github-corner:hover .octo-arm{animation:none}.github-corner .octo-arm{animation:octocat-wave 560ms ease-in-out}}</style>
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,4 @@
|
|||||||
{% extends "layout.html" %}
|
{% extends "layout.html" %}
|
||||||
{% set active_page = "master_list" %}
|
|
||||||
{% block body %}
|
{% block body %}
|
||||||
|
|
||||||
<hr />
|
<hr />
|
||||||
|
|||||||
@@ -1,51 +0,0 @@
|
|||||||
<div class="modal fade" id="myModal" tabindex="-1" role="dialog" aria-labelledby="myModalLabel">
|
|
||||||
|
|
||||||
<form id="myModalForm" method="post">
|
|
||||||
|
|
||||||
<div class="modal-dialog" role="document">
|
|
||||||
<div id="myModal-content" class="modal-content">
|
|
||||||
<div id="myModal-header" class="modal-header">
|
|
||||||
<button type="button" class="close" data-dismiss="modal" aria-label="Close">
|
|
||||||
<span aria-hidden="true">×</span>
|
|
||||||
</button>
|
|
||||||
<h4 class="modal-title" id="myModalLabel">
|
|
||||||
Send us feedback!
|
|
||||||
</h4>
|
|
||||||
</div>
|
|
||||||
<div id="myModal-body" class="modal-body">
|
|
||||||
<div id="modal-feedback-smile-frown-container" class="container-fluid">
|
|
||||||
<div id="modal-feedback-smile-div" class="col-xs-6 text-center"
|
|
||||||
onClick="smile()">
|
|
||||||
<i id="modal-feedback-smile-icon" class="fa fa-smile-o fa-4x" aria-hidden="true"></i>
|
|
||||||
</div>
|
|
||||||
<div id="modal-feedback-frown-div" class="col-xs-6 text-center"
|
|
||||||
onClick="frown()">
|
|
||||||
<i id="modal-feedback-frown-icon" class="fa fa-frown-o fa-4x" aria-hidden="true"></i>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div class="container-fluid">
|
|
||||||
<p> </p>
|
|
||||||
</div>
|
|
||||||
<div id="modal-feedback-textarea-container" class="container-fluid">
|
|
||||||
<textarea id="modal-feedback-textarea" rows="6"></textarea>
|
|
||||||
</div>
|
|
||||||
<div id="modal-too-long" class="container-fluid" >
|
|
||||||
<p id="modal-too-long-text" class="lead">Please limit the length of your feedback. Thank you in advance!</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div id="myModal-footer" class="modal-footer">
|
|
||||||
<div class="text-center">
|
|
||||||
<button id="submit-feedback-btn" type="button"
|
|
||||||
onClick="submit_feedback()"
|
|
||||||
class="btn btn-lg btn-primary">
|
|
||||||
Send
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</form>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
@@ -1,8 +1,7 @@
|
|||||||
{% extends "layout.html" %}
|
{% extends "layout.html" %}
|
||||||
{% set active_page = "search" %}
|
|
||||||
{% block body %}
|
{% block body %}
|
||||||
|
|
||||||
<div id="search-bar-container" class="container">
|
<div class="container">
|
||||||
<div class="row">
|
<div class="row">
|
||||||
|
|
||||||
<div class="col-xs-12">
|
<div class="col-xs-12">
|
||||||
@@ -13,11 +12,7 @@
|
|||||||
<p><button id="the-big-one" type="submit" style="font-size: 20px; padding: 10px; padding-left: 50px; padding-right: 50px;"
|
<p><button id="the-big-one" type="submit" style="font-size: 20px; padding: 10px; padding-left: 50px; padding-right: 50px;"
|
||||||
value="search" class="btn btn-primary">Search</button>
|
value="search" class="btn btn-primary">Search</button>
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
{% if parsed_query %}
|
|
||||||
<p><a href="{{ url_for('search')}}?query=&fields=">[clear all results]</a>
|
<p><a href="{{ url_for('search')}}?query=&fields=">[clear all results]</a>
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
</p>
|
</p>
|
||||||
</form>
|
</form>
|
||||||
</center>
|
</center>
|
||||||
@@ -25,11 +20,18 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div style="height: 20px;"><p> </p></div>
|
<div class="container">
|
||||||
|
|
||||||
<div id="info-bars-container" class="container">
|
|
||||||
<div class="row">
|
<div class="row">
|
||||||
|
|
||||||
|
{% if directories %}
|
||||||
|
<div class="col-xs-12 info directories-cloud">
|
||||||
|
<b>File directories:</b>
|
||||||
|
{% for d in directories %}
|
||||||
|
<a href="{{url_for('search')}}?query={{d|trim}}&fields=filename">{{d|trim}}</a>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
<ul class="list-group">
|
<ul class="list-group">
|
||||||
|
|
||||||
{% if config['SHOW_PARSED_QUERY'] and parsed_query %}
|
{% if config['SHOW_PARSED_QUERY'] and parsed_query %}
|
||||||
@@ -44,9 +46,6 @@
|
|||||||
</li>
|
</li>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
|
||||||
|
|
||||||
{# use "if parsed_query" to check if this is
|
|
||||||
a new search or search results #}
|
|
||||||
{% if parsed_query %}
|
{% if parsed_query %}
|
||||||
<li class="list-group-item">
|
<li class="list-group-item">
|
||||||
<div class="container-fluid">
|
<div class="container-fluid">
|
||||||
@@ -60,7 +59,6 @@
|
|||||||
</li>
|
</li>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
|
||||||
|
|
||||||
<li class="list-group-item">
|
<li class="list-group-item">
|
||||||
<div class="container-fluid">
|
<div class="container-fluid">
|
||||||
<div class="row">
|
<div class="row">
|
||||||
@@ -101,7 +99,7 @@
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
{% if parsed_query %}
|
{% if parsed_query %}
|
||||||
<div id="search-results-container" class="container">
|
<div class="container">
|
||||||
<div class="row">
|
<div class="row">
|
||||||
<table id="search-results" class="table">
|
<table id="search-results" class="table">
|
||||||
<thead id="search-results-header">
|
<thead id="search-results-header">
|
||||||
@@ -128,21 +126,44 @@
|
|||||||
{% if e.kind=="gdoc" %}
|
{% if e.kind=="gdoc" %}
|
||||||
{% if e.mimetype=="document" %}
|
{% if e.mimetype=="document" %}
|
||||||
<p><small>Drive Document</small</p>
|
<p><small>Drive Document</small</p>
|
||||||
|
<!--
|
||||||
|
<i class="fa fa-google fa-2x"></i>
|
||||||
|
<i class="fa fa-file-text fa-2x"></i>
|
||||||
|
-->
|
||||||
{% else %}
|
{% else %}
|
||||||
<p><small>Drive File</small</p>
|
<p><small>Drive File</small</p>
|
||||||
|
<!--
|
||||||
|
<i class="fa fa-google fa-2x"></i>
|
||||||
|
<i class="fa fa-file-o fa-2x"></i>
|
||||||
|
-->
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
|
||||||
{% elif e.kind=="issue" %}
|
{% elif e.kind=="issue" %}
|
||||||
<p><small>Issue</small</p>
|
<p><small>Issue</small</p>
|
||||||
|
<!--
|
||||||
|
<i class="fa fa-github fa-2x"></i>
|
||||||
|
<i class="fa fa-question fa-2x"></i>
|
||||||
|
-->
|
||||||
|
|
||||||
{% elif e.kind=="ghfile" %}
|
{% elif e.kind=="ghfile" %}
|
||||||
<p><small>Github File</small</p>
|
<p><small>Github File</small</p>
|
||||||
|
<!--
|
||||||
|
<i class="fa fa-github fa-2x"></i>
|
||||||
|
<i class="fa fa-file-o fa-2x"></i>
|
||||||
|
-->
|
||||||
|
|
||||||
{% elif e.kind=="markdown" %}
|
{% elif e.kind=="markdown" %}
|
||||||
<p><small>Github Markdown</small</p>
|
<p><small>Github Markdown</small</p>
|
||||||
|
<!--
|
||||||
|
<i class="fa fa-github fa-2x"></i>
|
||||||
|
<i class="fa fa-file-text-o fa-2x"></i>
|
||||||
|
-->
|
||||||
|
|
||||||
{% elif e.kind=="emailthread" %}
|
{% elif e.kind=="emailthread" %}
|
||||||
<p><small>Email Thread</small</p>
|
<p><small>Email Thread</small</p>
|
||||||
|
<!--
|
||||||
|
<i class="fa fa-envelope-o fa-2x"></i>
|
||||||
|
-->
|
||||||
|
|
||||||
{% else %}
|
{% else %}
|
||||||
<p><small>Unknown</small</p>
|
<p><small>Unknown</small</p>
|
||||||
|
|||||||
196
tests/Readme.md
196
tests/Readme.md
@@ -1,196 +0,0 @@
|
|||||||
Centillion Tests
|
|
||||||
================
|
|
||||||
|
|
||||||
Table of Contents
|
|
||||||
------------------
|
|
||||||
|
|
||||||
* [Centillion Tests](#centillion-tests)
|
|
||||||
* [Test Plan](#test-plan)
|
|
||||||
* [Local Tests](#local-tests)
|
|
||||||
* [Short Tests](#short-tests)
|
|
||||||
* [Long Tests](#long-tests)
|
|
||||||
* [Credentials](#credentials)
|
|
||||||
* [Detailed Description of Tests](#detailed-description-of-tests)
|
|
||||||
* [Authentication Layer Tests](#authentication-layer-tests)
|
|
||||||
* [Search Function Tests](#search-function-tests)
|
|
||||||
* [Master List Endpoint Tests](#master-list-endpoint-tests)
|
|
||||||
* [Control Panel Endpoint Tests](#control-panel-endpoint-tests)
|
|
||||||
* [Continuous Integration Plan](#continuous-integration-plan)
|
|
||||||
* [Procedure/Checklist](#procedurechecklist)
|
|
||||||
|
|
||||||
|
|
||||||
Test Plan
|
|
||||||
---------
|
|
||||||
|
|
||||||
Related: <https://github.com/dcppc/centillion/issues/82>
|
|
||||||
|
|
||||||
The test suite for centillion needs to check each of the major
|
|
||||||
components of centillion, as well as check the authentication mechanism
|
|
||||||
using multiple login credentials.
|
|
||||||
|
|
||||||
We implement the following checks:
|
|
||||||
|
|
||||||
1. Check authentication mechanism(s) (yamasnake and florence python)
|
|
||||||
|
|
||||||
2. Check search function
|
|
||||||
|
|
||||||
3. Check master list endpoint
|
|
||||||
|
|
||||||
4. Check control panel endpoint
|
|
||||||
|
|
||||||
5. Check update search index endpoints
|
|
||||||
|
|
||||||
The tests are written such that the back end and front end are tested
|
|
||||||
separately.
|
|
||||||
|
|
||||||
We need also need different tiers of tests, so we don't max out API
|
|
||||||
calls by making lots of commits to multiple PRs.
|
|
||||||
|
|
||||||
We have three tiers of tests: \* Local tests - quick tests for CI, no
|
|
||||||
API calls \* Short tests - tests using dummy API accounts \* Long tests
|
|
||||||
- tests using DCPPC API accounts
|
|
||||||
|
|
||||||
### Local Tests
|
|
||||||
|
|
||||||
Local tests can be run locally without any interaction with APIs. These
|
|
||||||
will still utilize centillion's search schema, but will load the search
|
|
||||||
index with fake documents rather than fetching them from an API.
|
|
||||||
|
|
||||||
Uncle Archie, which runs CI tests, runs local tests only (unless you
|
|
||||||
request it to run short test or long test.)
|
|
||||||
|
|
||||||
### Short Tests
|
|
||||||
|
|
||||||
Short tests utilize credentials for bot accounts that have intentionally
|
|
||||||
been set up to have a "known" corpus of test documents. These would
|
|
||||||
provide unit-style tests for centillion - are the mechanics of indexing
|
|
||||||
a particular type of document from a particular API working?
|
|
||||||
|
|
||||||
### Long Tests
|
|
||||||
|
|
||||||
Long tests are indexing the real deal, utilizing the credentials used in
|
|
||||||
the final production centillion. This test takes longer but is more
|
|
||||||
likely to catch corner cases specific to the DCPPC documents.
|
|
||||||
|
|
||||||
Credentials
|
|
||||||
-----------
|
|
||||||
|
|
||||||
Running tests on centillion requires multiple sets of credentials. Let's
|
|
||||||
lay out what is needed:
|
|
||||||
|
|
||||||
- The Flask app requires a token/secret token API key pair to allow
|
|
||||||
users to authenticate through Github and confirm they are members of
|
|
||||||
the DCPPC organization. This OAuth application is owned by Charles
|
|
||||||
Reid (@charlesreid1).
|
|
||||||
|
|
||||||
- The search index needs a Github access token so that it can
|
|
||||||
interface with the Github API to index files and issues. This access
|
|
||||||
token is specified (along with other secrets) in the Flask
|
|
||||||
configuration file. The access key comes from Florence Python
|
|
||||||
(@fp9695253).
|
|
||||||
|
|
||||||
- The search index also requires a Google Drive API access token. This
|
|
||||||
must be an access token for a user who has authenticated with the
|
|
||||||
Centillion Google Drive OAuth application. This access token comes
|
|
||||||
from <mailroom@nihdatacommons.com>.
|
|
||||||
|
|
||||||
- The search index requires API credentials for any other APIs
|
|
||||||
associated with other document collections (Groups.io, Hypothesis,
|
|
||||||
Disqus).
|
|
||||||
|
|
||||||
- The backend test requires the credentials provided to Flask.
|
|
||||||
|
|
||||||
- The frontend test (Selenium) needs two Github username/passwords:
|
|
||||||
one for Florence Python (@fp9695253) and one for Yamma Snake
|
|
||||||
(@yammasnake). These are required to simulate the user
|
|
||||||
authenticating with Github through the browser.
|
|
||||||
- The frontend test credentials are a special case.
|
|
||||||
- The frontend tests expect credentials to come from environment
|
|
||||||
variables.
|
|
||||||
- These environment variables get passed in at test time.
|
|
||||||
- Tests are all run on [Uncle
|
|
||||||
Archie](https://github.com/dcppc/uncle-archie).
|
|
||||||
- Uncle Archie already has to protect a confidential config file
|
|
||||||
containing Github credentials, so add additional credentials for
|
|
||||||
frontend tests there.
|
|
||||||
- Logical separation: these credentials are not needed to
|
|
||||||
*operate* centillion, these credentials are needed to *test*
|
|
||||||
centillion
|
|
||||||
- Uncle Archie already requires github credentials, already
|
|
||||||
protects sensitive info.
|
|
||||||
- Google Drive requiring its own credentials file on disk is a
|
|
||||||
pain.
|
|
||||||
|
|
||||||
In summary: tests use the `config_flask.py` and `config_centillion.py`
|
|
||||||
files to provide it with the API keys it needs and to instruct it on
|
|
||||||
what to index. The credentials and config files will control what the
|
|
||||||
search index will actually index. The Uncle Archie CI tester config file
|
|
||||||
contains the credentials needed to run frontend tests (check the
|
|
||||||
login/authentication layer).
|
|
||||||
|
|
||||||
Detailed Description of Tests
|
|
||||||
-----------------------------
|
|
||||||
|
|
||||||
### Authentication Layer Tests
|
|
||||||
|
|
||||||
Frontend tests run as Florence Python:
|
|
||||||
|
|
||||||
- Can we log in via github and reach centillion
|
|
||||||
- Can we reach the control panel
|
|
||||||
|
|
||||||
Frontend tests run as Yamma Snake (DCPPC member):
|
|
||||||
|
|
||||||
- Can we log in via github and reach centillion
|
|
||||||
- Can we reach the control panel
|
|
||||||
|
|
||||||
### Search Function Tests
|
|
||||||
|
|
||||||
Frontend tests:
|
|
||||||
|
|
||||||
- Can we enter something into search box and submit
|
|
||||||
- Can we sort the results
|
|
||||||
- Do the results look okay
|
|
||||||
|
|
||||||
Backend tests:
|
|
||||||
|
|
||||||
- Load the search index and run a query using whoosh API
|
|
||||||
|
|
||||||
### Master List Endpoint Tests
|
|
||||||
|
|
||||||
Frontend tests:
|
|
||||||
|
|
||||||
- Can we get to the master list page
|
|
||||||
- Can we sort the results
|
|
||||||
- Do the results look okay
|
|
||||||
|
|
||||||
Backend tests:
|
|
||||||
|
|
||||||
- Check the output of the `/list` API endpoint
|
|
||||||
|
|
||||||
### Control Panel Endpoint Tests
|
|
||||||
|
|
||||||
Frontend tests:
|
|
||||||
|
|
||||||
- Can we get to the control panel page
|
|
||||||
- Can we click the button to trigger an indexing event
|
|
||||||
|
|
||||||
Backend tests:
|
|
||||||
|
|
||||||
- Trigger a re-index of the search index from the backend.
|
|
||||||
|
|
||||||
### Continuous Integration Plan
|
|
||||||
|
|
||||||
Tests are automatically run using Uncle Archie for continuous
|
|
||||||
integration and deployment.
|
|
||||||
|
|
||||||
Procedure/Checklist
|
|
||||||
-------------------
|
|
||||||
|
|
||||||
Pre-release procedure:
|
|
||||||
|
|
||||||
- prepare to run all test
|
|
||||||
|
|
||||||
- run short tests
|
|
||||||
- deploy to beta
|
|
||||||
- run long tests
|
|
||||||
- test out
|
|
||||||
Reference in New Issue
Block a user