30 Commits

Author SHA1 Message Date
ab76226b0c Merge pull request #90 from dcppc/add-dates-and-subgroups-to-emails
Add dates and subgroups to emails
2018-08-24 00:07:40 -07:00
a4ebef6e6f extract date and time from email threads pages 2018-08-24 00:04:35 -07:00
bad50efa9b add groups and tags to schema; update how we determine timestamps; handle exceptions when we add the document to the writer, rather than elsewhere 2018-08-24 00:03:23 -07:00
629fc063db move where exception is caught (exception was also incorrect.) 2018-08-24 00:01:26 -07:00
3b0baa21de switched created_time, modified_time, indexed_time over to DATETIME. added DateParserPlugin to query QueryParser. added time fields to those being searched by default. tests do not seem to be working. 2018-08-23 19:01:40 -07:00
6bfadef829 Merge pull request #73 from dcppc/feedback-floater
Add a feedback mechanism
2018-08-21 11:33:34 -07:00
c38683ae9f (resolve conflict) Merge branch 'dcppc' into feedback-floater
* dcppc:
  add centillion config back. no sensitive info.
  add option to set port at runtime with CENTILLION_PORT environment variable
  add a bit o whitespace
2018-08-21 11:32:59 -07:00
3f5349a5a6 Merge pull request #80 from dcppc/add-centillion-config-back
add centillion config back. no sensitive info.
2018-08-21 11:16:21 -07:00
f88cf6ecad add centillion config back. no sensitive info. 2018-08-21 11:15:29 -07:00
ec54292a4b Merge pull request #79 from dcppc/add-port-env-var
add option to set port at runtime
2018-08-21 11:12:17 -07:00
296132d356 add option to set port at runtime with CENTILLION_PORT environment variable 2018-08-21 11:09:46 -07:00
0bc40ba323 Merge pull request #76 from dcppc/add-whitespace
add a bit o whitespace
2018-08-21 10:33:20 -07:00
8143e214c2 add a bit o whitespace 2018-08-21 10:06:16 -07:00
b015da2e9b add dismissable "thanks for your feedback" message to top 2018-08-20 20:42:58 -07:00
9c6b57ba85 improve message formatting 2018-08-20 15:04:21 -07:00
a080eebc29 add dumy function as placeholder for where we add info messages 2018-08-20 15:04:03 -07:00
323d7ce8ca return better messages 2018-08-20 15:03:21 -07:00
da62a5c887 add successful post call and export to JSON db 2018-08-20 14:10:20 -07:00
2714ad3e0c update todo 2018-08-20 14:09:58 -07:00
5e1388e8a8 move modal into its own .html file 2018-08-20 10:34:20 -07:00
f40cccac99 update todo with tasks 2018-08-20 10:33:51 -07:00
c72fc44ea7 fix button and smiley styles 2018-08-20 10:33:36 -07:00
cf417917c9 add /feedback post route 2018-08-20 10:29:40 -07:00
8aaad93e68 Merge remote-tracking branch 'origin/dcppc' into feedback-floater
* origin/dcppc:
  remove copper requirement for /list endpoint
2018-08-19 21:05:22 -07:00
bf8d99c732 Merge pull request #71 from dcppc/hotfix/list-endpoint
remove copper requirement for /list endpoint
2018-08-19 20:49:12 -07:00
20c55891f3 remove copper requirement for /list endpoint 2018-08-19 20:44:06 -07:00
685058545a feedback button successfully triggers a modal 2018-08-19 01:37:55 -07:00
23fd17132e add page self-identifiers. add "send feedback" button. fix layouts. 2018-08-19 01:16:13 -07:00
d3ba1f11a7 Merge pull request #63 from dcppc/hotfix/search-box
fix "Search Metadata" label so search does not break
2018-08-17 08:35:16 -07:00
36e89fbc65 fix "Search Metadata" label so search does not break 2018-08-17 08:31:15 -07:00
21 changed files with 723 additions and 348 deletions

2
.gitignore vendored
View File

@@ -1,4 +1,4 @@
config_centillion.py feedback_database.json
config_flask.py config_flask.py
vp vp
credentials.json credentials.json

View File

@@ -12,8 +12,13 @@ one centillion is 3.03 log-times better than a googol.
## What Is It ## What Is It
Centillion (https://github.com/dcppc/centillion) is a search engine that can index Centillion (https://github.com/dcppc/centillion) is a search engine that can index
three kinds of collections: Google Documents (.docx files), Github issues, and Markdown files in different kinds of document collections: Google Documents (.docx files), Google Drive files,
Github repos. Github issues, Github files, Github Markdown files, and Groups.io email threads.
## What Is It
We define the types of documents the centillion should index, We define the types of documents the centillion should index,
what info and how. The centillion then builds and what info and how. The centillion then builds and

56
Todo.md
View File

@@ -1,47 +1,29 @@
# todo # todo
Main task: ux improvements:
- hashing and caching - feedback tools
- <s>first, working out the logic of how we group items into sets - integrating master list into single list
- needs to be deleted - providing advanced search interfce
- needs to be updated
- needs to be added
- for docs, issues, and comments</s>
- second, when we add or update an item, need to:
- go through the motions, download file, extract text
- check for existing indexed doc with that id
- check if existing indexed doc has same hash
- if so, skip
- otherwise, delete and re-index
Other bugs: big picture improvements:
- Some github issues have no title (?) - hypothesis API
- <s>Need to combine issues with comments</s> - folksonomy tagging with hypothesis
- Not able to index markdown files _in a repo_ - tags, expanded schema
- (Longer term) update main index vs update diff index
Needs:
- <s>control panel</s>
Thursday product:
- Everything re-indexed nightly
- Search engine built on all documents in Google Drive, all issues, markdown files
- Using pandoc to extract Google Drive document contents
- BRIEF quickstart documentation
Future:
- Future plans to improve - plugins, improving matching
- Subdomain plans
- Folksonomy tagging and integration plans
config options for plugins feedback form: where we are at
conditional blocks with import github inside - feedback button
complicated tho - better to have components split off - button triggers modal form
- modal has emojis for feedback, text box, buttons
- clicking emojis changes color, to select
- clicking submit with filled out form submits to an endpoint
- clicking submit also closes form, but only if submit successful
feedback form: what we need to do
- fix alerts - thank you for your feedback doesn't show up until a refresh
- probably an easy ajax fix

View File

@@ -3,6 +3,7 @@ import subprocess
import codecs import codecs
import os, json import os, json
from datetime import datetime
from werkzeug.contrib.fixers import ProxyFix from werkzeug.contrib.fixers import ProxyFix
from flask import Flask, request, redirect, url_for, render_template, flash, jsonify from flask import Flask, request, redirect, url_for, render_template, flash, jsonify
@@ -262,13 +263,57 @@ def list_docs(doctype):
all_orgs = resp.json() all_orgs = resp.json()
for org in all_orgs: for org in all_orgs:
if org['login']=='dcppc': if org['login']=='dcppc':
copper_team_id = '2700235'
mresp = github.get('/teams/%s/members/%s'%(copper_team_id,username))
if mresp.status_code==204:
# Business as usual # Business as usual
search = Search(app.config["INDEX_DIR"]) search = Search(app.config["INDEX_DIR"])
return jsonify(search.get_list(doctype)) return jsonify(search.get_list(doctype))
# nope
return render_template('403.html')
@app.route('/feedback', methods=['POST'])
def parse_request():
if not github.authorized:
return redirect(url_for("github.login"))
username = github.get("/user").json()['login']
resp = github.get("/user/orgs")
if resp.ok:
all_orgs = resp.json()
for org in all_orgs:
if org['login']=='dcppc':
try:
# Business as usual
data = request.form.to_dict();
data['github_login'] = username
data['timestamp'] = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
feedback_database = 'feedback_database.json'
if not os.path.isfile(feedback_database):
with open(feedback_database,'w') as f:
json_data = [data]
json.dump(json_data, f, indent=4)
else:
json_data = []
with open(feedback_database,'r') as f:
json_data = json.load(f)
json_data.append(data)
with open(feedback_database,'w') as f:
json.dump(json_data, f, indent=4)
## Should be done with Javascript
#flash("Thank you for your feedback!")
return jsonify({'status':'ok','message':'Thank you for your feedback!'})
except:
return jsonify({'status':'error','message':'An error was encountered while submitting your feedback. Try submitting an issue in the <a href="https://github.com/dcppc/centillion/issues/new">dcppc/centillion</a> repository.'})
# nope
return render_template('403.html') return render_template('403.html')
@app.errorhandler(404) @app.errorhandler(404)
@@ -297,5 +342,10 @@ def store_search(query, fields):
if __name__ == '__main__': if __name__ == '__main__':
# if running local instance, set to true # if running local instance, set to true
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = 'true' os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = 'true'
app.run(host="0.0.0.0",port=5000) port = os.environ.get('CENTILLION_PORT','')
if port=='':
port = 5000
else:
port = int(port)
app.run(host="0.0.0.0",port=port)

View File

@@ -21,6 +21,8 @@ import dateutil.parser
from whoosh.qparser import MultifieldParser, QueryParser from whoosh.qparser import MultifieldParser, QueryParser
from whoosh.analysis import StemmingAnalyzer from whoosh.analysis import StemmingAnalyzer
from whoosh.qparser.dateparse import DateParserPlugin
from whoosh import fields, index
""" """
@@ -180,30 +182,38 @@ class Search:
# is defined. # is defined.
schema = Schema( schema = Schema(
id = ID(stored=True, unique=True), id = fields.ID(stored=True, unique=True),
kind = ID(stored=True), kind = fields.ID(stored=True),
created_time = ID(stored=True), created_time = fields.DATETIME(stored=True),
modified_time = ID(stored=True), modified_time = fields.DATETIME(stored=True),
indexed_time = ID(stored=True), indexed_time = fields.DATETIME(stored=True),
title = TEXT(stored=True, field_boost=100.0), title = fields.TEXT(stored=True, field_boost=100.0),
url = ID(stored=True, unique=True),
mimetype=ID(stored=True), url = fields.ID(stored=True),
owner_email=ID(stored=True),
owner_name=TEXT(stored=True),
repo_name=TEXT(stored=True), mimetype = fields.TEXT(stored=True),
repo_url=ID(stored=True),
github_user=TEXT(stored=True), owner_email = fields.ID(stored=True),
owner_name = fields.TEXT(stored=True),
# mainly for email threads, groups.io, hypothesis
group = fields.ID(stored=True),
repo_name = fields.TEXT(stored=True),
repo_url = fields.ID(stored=True),
github_user = fields.TEXT(stored=True),
tags = fields.KEYWORD(commas=True,
stored=True,
lowercase=True),
# comments only # comments only
issue_title=TEXT(stored=True, field_boost=100.0), issue_title = fields.TEXT(stored=True, field_boost=100.0),
issue_url=ID(stored=True), issue_url = fields.ID(stored=True),
content=TEXT(stored=True, analyzer=stemming_analyzer) content = fields.TEXT(stored=True, analyzer=stemming_analyzer)
) )
@@ -243,17 +253,22 @@ class Search:
writer.delete_by_term('id',item['id']) writer.delete_by_term('id',item['id'])
# Index a plain google drive file # Index a plain google drive file
created_time = dateutil.parser.parse(item['createdTime'])
modified_time = dateutil.parser.parse(item['modifiedTime'])
indexed_time = datetime.now().replace(microsecond=0)
try:
writer.add_document( writer.add_document(
id = item['id'], id = item['id'],
kind = 'gdoc', kind = 'gdoc',
created_time = item['createdTime'], created_time = created_time,
modified_time = item['modifiedTime'], modified_time = modified_time,
indexed_time = datetime.now().replace(microsecond=0).isoformat(), indexed_time = indexed_time,
title = item['name'], title = item['name'],
url = item['webViewLink'], url = item['webViewLink'],
mimetype = mimetype, mimetype = mimetype,
owner_email = item['owners'][0]['emailAddress'], owner_email = item['owners'][0]['emailAddress'],
owner_name = item['owners'][0]['displayName'], owner_name = item['owners'][0]['displayName'],
group='',
repo_name='', repo_name='',
repo_url='', repo_url='',
github_user='', github_user='',
@@ -261,6 +276,9 @@ class Search:
issue_url='', issue_url='',
content = content content = content
) )
except ValueError as e:
print(repr(e))
print(" > XXXXXX Failed to index Google Drive file \"%s\""%(item['name']))
else: else:
@@ -314,7 +332,7 @@ class Search:
) )
assert output == "" assert output == ""
except RuntimeError: except RuntimeError:
print(" > XXXXXX Failed to index document \"%s\""%(item['name'])) print(" > XXXXXX Failed to index Google Drive document \"%s\""%(item['name']))
# If export was successful, read contents of markdown # If export was successful, read contents of markdown
@@ -342,17 +360,22 @@ class Search:
else: else:
print(" > Creating a new record") print(" > Creating a new record")
try:
created_time = dateutil.parser.parse(item['createdTime'])
modified_time = dateutil.parser.parse(item['modifiedTime'])
indexed_time = datetime.now()
writer.add_document( writer.add_document(
id = item['id'], id = item['id'],
kind = 'gdoc', kind = 'gdoc',
created_time = item['createdTime'], created_time = created_time,
modified_time = item['modifiedTime'], modified_time = modified_time,
indexed_time = datetime.now().replace(microsecond=0).isoformat(), indexed_time = indexed_time,
title = item['name'], title = item['name'],
url = item['webViewLink'], url = item['webViewLink'],
mimetype = mimetype, mimetype = mimetype,
owner_email = item['owners'][0]['emailAddress'], owner_email = item['owners'][0]['emailAddress'],
owner_name = item['owners'][0]['displayName'], owner_name = item['owners'][0]['displayName'],
group='',
repo_name='', repo_name='',
repo_url='', repo_url='',
github_user='', github_user='',
@@ -360,6 +383,10 @@ class Search:
issue_url='', issue_url='',
content = content content = content
) )
except ValueError as e:
print(repr(e))
print(" > XXXXXX Failed to index Google Drive file \"%s\""%(item['name']))
@@ -393,13 +420,14 @@ class Search:
issue_comment_content += comment.body.rstrip() issue_comment_content += comment.body.rstrip()
issue_comment_content += "\n" issue_comment_content += "\n"
# Now create the actual search index record # Now create the actual search index record.
created_time = clean_timestamp(issue.created_at)
modified_time = clean_timestamp(issue.updated_at)
indexed_time = clean_timestamp(datetime.now())
# Add one document per issue thread, # Add one document per issue thread,
# containing entire text of thread. # containing entire text of thread.
created_time = issue.created_at
modified_time = issue.updated_at
indexed_time = datetime.now()
try:
writer.add_document( writer.add_document(
id = issue.html_url, id = issue.html_url,
kind = 'issue', kind = 'issue',
@@ -411,6 +439,7 @@ class Search:
mimetype='', mimetype='',
owner_email='', owner_email='',
owner_name='', owner_name='',
group='',
repo_name = repo_name, repo_name = repo_name,
repo_url = repo_url, repo_url = repo_url,
github_user = issue.user.login, github_user = issue.user.login,
@@ -418,6 +447,9 @@ class Search:
issue_url = issue.html_url, issue_url = issue.html_url,
content = issue_comment_content content = issue_comment_content
) )
except ValueError as e:
print(repr(e))
print(" > XXXXXX Failed to index Github issue \"%s\""%(issue.title))
@@ -447,7 +479,8 @@ class Search:
print(" > XXXXXXXX Failed to find file info.") print(" > XXXXXXXX Failed to find file info.")
return return
indexed_time = clean_timestamp(datetime.now())
indexed_time = datetime.now()
if fext in MARKDOWN_EXTS: if fext in MARKDOWN_EXTS:
print("Indexing markdown doc %s from repo %s"%(fname,repo_name)) print("Indexing markdown doc %s from repo %s"%(fname,repo_name))
@@ -476,17 +509,19 @@ class Search:
usable_url = "https://github.com/%s/blob/master/%s"%(repo_name, fpath) usable_url = "https://github.com/%s/blob/master/%s"%(repo_name, fpath)
# Now create the actual search index record # Now create the actual search index record
try:
writer.add_document( writer.add_document(
id = fsha, id = fsha,
kind = 'markdown', kind = 'markdown',
created_time = '', created_time = None,
modified_time = '', modified_time = None,
indexed_time = indexed_time, indexed_time = indexed_time,
title = fname, title = fname,
url = usable_url, url = usable_url,
mimetype='', mimetype='',
owner_email='', owner_email='',
owner_name='', owner_name='',
group='',
repo_name = repo_name, repo_name = repo_name,
repo_url = repo_url, repo_url = repo_url,
github_user = '', github_user = '',
@@ -494,6 +529,11 @@ class Search:
issue_url = '', issue_url = '',
content = content content = content
) )
except ValueError as e:
print(repr(e))
print(" > XXXXXX Failed to index Github markdown file \"%s\""%(fname))
else: else:
print("Indexing github file %s from repo %s"%(fname,repo_name)) print("Indexing github file %s from repo %s"%(fname,repo_name))
@@ -501,17 +541,19 @@ class Search:
key = fname+"_"+fsha key = fname+"_"+fsha
# Now create the actual search index record # Now create the actual search index record
try:
writer.add_document( writer.add_document(
id = key, id = key,
kind = 'ghfile', kind = 'ghfile',
created_time = '', created_time = None,
modified_time = '', modified_time = None,
indexed_time = indexed_time, indexed_time = indexed_time,
title = fname, title = fname,
url = repo_url, url = repo_url,
mimetype='', mimetype='',
owner_email='', owner_email='',
owner_name='', owner_name='',
group='',
repo_name = repo_name, repo_name = repo_name,
repo_url = repo_url, repo_url = repo_url,
github_user = '', github_user = '',
@@ -519,6 +561,9 @@ class Search:
issue_url = '', issue_url = '',
content = '' content = ''
) )
except ValueError as e:
print(repr(e))
print(" > XXXXXX Failed to index Github file \"%s\""%(fname))
@@ -532,20 +577,32 @@ class Search:
Use a Github file API record to add a filename Use a Github file API record to add a filename
to the search index. to the search index.
""" """
indexed_time = clean_timestamp(datetime.now()) if 'created_time' in d.keys() and d['created_time'] is not None:
created_time = d['created_time']
else:
created_time = None
if 'modified_time' in d.keys() and d['modified_time'] is not None:
modified_time = d['modified_time']
else:
modified_time = None
indexed_time = datetime.now()
# Now create the actual search index record # Now create the actual search index record
try:
writer.add_document( writer.add_document(
id = d['permalink'], id = d['permalink'],
kind = 'emailthread', kind = 'emailthread',
created_time = '', created_time = created_time,
modified_time = '', modified_time = modified_time,
indexed_time = indexed_time, indexed_time = indexed_time,
title = d['subject'], title = d['subject'],
url = d['permalink'], url = d['permalink'],
mimetype='', mimetype='',
owner_email='', owner_email='',
owner_name=d['original_sender'], owner_name=d['original_sender'],
group=d['subgroup'],
repo_name = '', repo_name = '',
repo_url = '', repo_url = '',
github_user = '', github_user = '',
@@ -553,7 +610,9 @@ class Search:
issue_url = '', issue_url = '',
content = d['content'] content = d['content']
) )
except ValueError as e:
print(repr(e))
print(" > XXXXXX Failed to index Groups.io thread \"%s\""%(d['subject']))
@@ -631,10 +690,10 @@ class Search:
full_items[f['id']] = f full_items[f['id']] = f
## Shorter: ## Shorter:
#break
# Longer:
if nextPageToken is None:
break break
## Longer:
#if nextPageToken is None:
# break
writer = self.ix.writer() writer = self.ix.writer()
@@ -642,7 +701,7 @@ class Search:
temp_dir = tempfile.mkdtemp(dir=os.getcwd()) temp_dir = tempfile.mkdtemp(dir=os.getcwd())
print("Temporary directory: %s"%(temp_dir)) print("Temporary directory: %s"%(temp_dir))
try:
# Drop any id in indexed_ids # Drop any id in indexed_ids
# not in remote_ids # not in remote_ids
@@ -670,6 +729,13 @@ class Search:
self.add_drive_file(writer, item, temp_dir, config, update=False) self.add_drive_file(writer, item, temp_dir, config, update=False)
count += 1 count += 1
except Exception as e:
print("ERROR: While adding Google Drive files to search index")
print("-"*40)
print(repr(e))
print("-"*40)
print("Continuing...")
pass
print("Cleaning temporary directory: %s"%(temp_dir)) print("Cleaning temporary directory: %s"%(temp_dir))
subprocess.call(['rm','-fr',temp_dir]) subprocess.call(['rm','-fr',temp_dir])
@@ -1074,7 +1140,7 @@ class Search:
elif doctype=='issue': elif doctype=='issue':
item_keys = ['title','repo_name','repo_url','url','created_time','modified_time'] item_keys = ['title','repo_name','repo_url','url','created_time','modified_time']
elif doctype=='emailthread': elif doctype=='emailthread':
item_keys = ['title','owner_name','url'] item_keys = ['title','owner_name','url','created_time','modified_time']
elif doctype=='ghfile': elif doctype=='ghfile':
item_keys = ['title','repo_name','repo_url','url'] item_keys = ['title','repo_name','repo_url','url']
elif doctype=='markdown': elif doctype=='markdown':
@@ -1091,10 +1157,6 @@ class Search:
for r in results: for r in results:
d = {} d = {}
for k in item_keys: for k in item_keys:
if k=='created_time' or k=='modified_time':
#d[k] = r[k]
d[k] = dateutil.parser.parse(r[k]).strftime("%Y-%m-%d")
else:
d[k] = r[k] d[k] = r[k]
json_results.append(d) json_results.append(d)
@@ -1108,7 +1170,9 @@ class Search:
query_string = " ".join(query_list) query_string = " ".join(query_list)
query = None query = None
if ":" in query_string: if ":" in query_string:
query = QueryParser("content", self.schema).parse(query_string) query = QueryParser("content", self.schema)
query.add_plugin(DateParserPlugin(free=True))
query = query.parse(query_string)
elif len(fields) == 1 and fields[0] == "filename": elif len(fields) == 1 and fields[0] == "filename":
pass pass
elif len(fields) == 2: elif len(fields) == 2:
@@ -1116,9 +1180,12 @@ class Search:
else: else:
# If the user does not specify a field, # If the user does not specify a field,
# these are the fields that are actually searched # these are the fields that are actually searched
fields = ['title', 'content','owner_name','owner_email','url'] fields = ['title', 'content','owner_name','owner_email','url','created_date','modified_date']
if not query: if not query:
query = MultifieldParser(fields, schema=self.ix.schema).parse(query_string) query = MultifieldParser(fields, schema=self.ix.schema)
query.add_plugin(DateParserPlugin(free=True))
query = query.parse(query_string)
#query = MultifieldParser(fields, schema=self.ix.schema).parse(query_string)
parsed_query = "%s" % query parsed_query = "%s" % query
print("query: %s" % parsed_query) print("query: %s" % parsed_query)
results = searcher.search(query, terms=False, scored=True, groupedby="kind") results = searcher.search(query, terms=False, scored=True, groupedby="kind")

28
config_centillion.py Normal file
View File

@@ -0,0 +1,28 @@
config = {
"repositories" : [
"dcppc/project-management",
"dcppc/nih-demo-meetings",
"dcppc/internal",
"dcppc/organize",
"dcppc/dcppc-bot",
"dcppc/full-stacks",
"dcppc/design-guidelines-discuss",
"dcppc/dcppc-deliverables",
"dcppc/dcppc-milestones",
"dcppc/crosscut-metadata",
"dcppc/lucky-penny",
"dcppc/dcppc-workshops",
"dcppc/metadata-matrix",
"dcppc/data-stewards",
"dcppc/dcppc-phase1-demos",
"dcppc/apis",
"dcppc/2018-june-workshop",
"dcppc/2018-july-workshop",
"dcppc/2018-august-workshop",
"dcppc/2018-september-workshop",
"dcppc/design-guidelines",
"dcppc/2018-may-workshop",
"dcppc/centillion"
]
}

View File

@@ -1,5 +1,7 @@
import requests, os, re import requests, os, re
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
import dateutil.parser
import datetime
class GroupsIOException(Exception): class GroupsIOException(Exception):
pass pass
@@ -64,7 +66,7 @@ class GroupsIOArchivesCrawler(object):
## Short circuit ## Short circuit
## for debugging purposes ## for debugging purposes
#break break
return subgroups return subgroups
@@ -251,7 +253,7 @@ class GroupsIOArchivesCrawler(object):
subject = soup.find('title').text subject = soup.find('title').text
# Extract information for the schema: # Extract information for the schema:
# - permalink for thread (done) # - permalink for thread (done above)
# - subject/title (done) # - subject/title (done)
# - original sender email/name (done) # - original sender email/name (done)
# - content (done) # - content (done)
@@ -266,11 +268,35 @@ class GroupsIOArchivesCrawler(object):
pass pass
else: else:
# found an email! # found an email!
# this is a maze, thanks groups.io # this is a maze, not amazing.
# thanks groups.io!
td = tr.find('td') td = tr.find('td')
divrow = td.find('div',{'class':'row'}).find('div',{'class':'pull-left'})
sender_divrow = td.find('div',{'class':'row'})
sender_divrow = sender_divrow.find('div',{'class':'pull-left'})
if (i+1)==1: if (i+1)==1:
original_sender = divrow.text.strip() original_sender = sender_divrow.text.strip()
date_divrow = td.find('div',{'class':'row'})
date_divrow = date_divrow.find('div',{'class':'pull-right'})
date_divrow = date_divrow.find('font',{'class':'text-muted'})
date_divrow = date_divrow.find('script').text
try:
time_seconds = re.search(' [0-9]{1,} ',date_divrow).group(0)
time_seconds = time_seconds.strip()
# Thanks groups.io for the weird date formatting
time_seconds = time_seconds[:10]
mmicro_seconds = time_seconds[10:]
if (i+1)==1:
created_time = datetime.datetime.utcfromtimestamp(int(time_seconds))
modified_time = datetime.datetime.utcfromtimestamp(int(time_seconds))
else:
modified_time = datetime.datetime.utcfromtimestamp(int(time_seconds))
except AttributeError:
created_time = None
modified_time = None
for div in td.find_all('div'): for div in td.find_all('div'):
if div.has_attr('id'): if div.has_attr('id'):
@@ -299,7 +325,10 @@ class GroupsIOArchivesCrawler(object):
thread = { thread = {
'permalink' : permalink, 'permalink' : permalink,
'created_time' : created_time,
'modified_time' : modified_time,
'subject' : subject, 'subject' : subject,
'subgroup' : subgroup_name,
'original_sender' : original_sender, 'original_sender' : original_sender,
'content' : full_content 'content' : full_content
} }
@@ -324,11 +353,13 @@ class GroupsIOArchivesCrawler(object):
results = [] results = []
for row in rows: for row in rows:
# We don't care about anything except title and ugly link # This is where we extract
# a list of thread titles
# and corresponding links.
subject = row.find('span',{'class':'subject'}) subject = row.find('span',{'class':'subject'})
title = subject.get_text() title = subject.get_text()
link = row.find('a')['href'] link = row.find('a')['href']
#print(title)
results.append((title,link)) results.append((title,link))
return results return results

133
static/feedback.js Normal file
View File

@@ -0,0 +1,133 @@
// submitting form with modal:
// https://stackoverflow.com/a/29068742
//
// closing a bootstrap modal with submit button:
// https://stackoverflow.com/a/33478107
//
// flask post data as json:
// https://stackoverflow.com/a/16664376
/* this function is called when the user submits
* the feedback form. it submits a post request
* to the flask server, which squirrels away the
* feedback in a file.
*/
function submit_feedback() {
// this function is called when submit button clicked
// algorithm:
// - check if text box has content
// - check if happy/sad filled out
var smile_active = $('#modal-feedback-smile-div').hasClass('smile-active');
var frown_active = $('#modal-feedback-frown-div').hasClass('frown-active');
if( !( smile_active || frown_active ) ) {
alert('Please pick the smile or the frown.')
} else if( $('#modal-feedback-textarea').val()=='' ) {
alert('Please provide us with some feedback.')
} else {
var user_sentiment = '';
if(smile_active) {
user_sentiment = 'smile';
} else {
user_sentiment = 'frown';
}
var escaped_text = $('#modal-feedback-textarea').val();
// prepare form data
var data = {
sentiment : user_sentiment,
content : escaped_text
};
// post the form. the callback function resets the form
$.post("/feedback",
data,
function(response) {
$('#myModal').modal('hide');
$('#myModalForm')[0].reset();
add_alert(response);
frown_unclick();
smile_unclick();
});
}
}
function add_alert(response) {
str = ""
str += '<div id="feedback-messages-container" class="container">';
if (response['status']=='ok') {
// if status is ok, use alert-success
str += ' <div id="feedback-messages-alert" class="alert alert-success alert-dismissible fade in">';
} else {
// otherwise use alert-danger
str += ' <div id="feedback-messages-alert" class="alert alert-danger alert-dismissible fade in">';
}
str += ' <a href="#" class="close" data-dismiss="alert" aria-label="close">&times;</a>';
str += ' <div id="feedback-messages-contianer" class="container-fluid">';
str += ' <div id="feedback-messages-div" class="co-xs-12">';
str += ' <p>'
str += response['message'];
str += ' </p>';
str += ' </div>';
str += ' </div>';
str += '</div>';
$('div#messages').append(str);
}
/* for those particularly wordy users... limit feedback to 1000 chars */
function cool_it() {
if($('#modal-feedback-textarea').val().length > 1100 ){
$('#modal-too-long').show();
} else {
$('#modal-too-long').hide();
}
}
/* smiley functions */
function smile_click() {
$('#modal-feedback-smile-div').addClass('smile-active');
$('#modal-feedback-smile-icon').addClass('smile-active');
}
function frown_click() {
$('#modal-feedback-frown-div').addClass('frown-active');
$('#modal-feedback-frown-icon').addClass('frown-active');
}
function smile_unclick() {
$('#modal-feedback-smile-div').removeClass('smile-active');
$('#modal-feedback-smile-icon').removeClass('smile-active');
}
function frown_unclick() {
$('#modal-feedback-frown-div').removeClass('frown-active');
$('#modal-feedback-frown-icon').removeClass('frown-active');
}
function smile() {
frown_unclick();
smile_click();
}
function frown() {
smile_unclick();
frown_click();
}
/* for those particularly wordy users... limit feedback to 1100 chars */
// how to check n characters in a textarea
// https://stackoverflow.com/a/19934613
/*
$(document).ready(function() {
$('#modal-feedback-textarea').on('change',function(event) {
if($('#modal-feedback-textarea').val().length > 1100 ){
$('#modal-too-long').show();
} else {
$('#modal-too-long').hide();
}
});
}
*/

View File

@@ -141,7 +141,7 @@ sSearch:"",bRegex:!1,bSmart:!0};m.models.oRow={nTr:null,anCells:null,_aData:[],_
sTitle:null,sType:null,sWidth:null,sWidthOrig:null};m.defaults={aaData:null,aaSorting:[[0,"asc"]],aaSortingFixed:[],ajax:null,aLengthMenu:[10,25,50,100],aoColumns:null,aoColumnDefs:null,aoSearchCols:[],asStripeClasses:null,bAutoWidth:!0,bDeferRender:!1,bDestroy:!1,bFilter:!0,bInfo:!0,bJQueryUI:!1,bLengthChange:!0,bPaginate:!0,bProcessing:!1,bRetrieve:!1,bScrollCollapse:!1,bServerSide:!1,bSort:!0,bSortMulti:!0,bSortCellsTop:!1,bSortClasses:!0,bStateSave:!1,fnCreatedRow:null,fnDrawCallback:null,fnFooterCallback:null, sTitle:null,sType:null,sWidth:null,sWidthOrig:null};m.defaults={aaData:null,aaSorting:[[0,"asc"]],aaSortingFixed:[],ajax:null,aLengthMenu:[10,25,50,100],aoColumns:null,aoColumnDefs:null,aoSearchCols:[],asStripeClasses:null,bAutoWidth:!0,bDeferRender:!1,bDestroy:!1,bFilter:!0,bInfo:!0,bJQueryUI:!1,bLengthChange:!0,bPaginate:!0,bProcessing:!1,bRetrieve:!1,bScrollCollapse:!1,bServerSide:!1,bSort:!0,bSortMulti:!0,bSortCellsTop:!1,bSortClasses:!0,bStateSave:!1,fnCreatedRow:null,fnDrawCallback:null,fnFooterCallback:null,
fnFormatNumber:function(a){return a.toString().replace(/\B(?=(\d{3})+(?!\d))/g,this.oLanguage.sThousands)},fnHeaderCallback:null,fnInfoCallback:null,fnInitComplete:null,fnPreDrawCallback:null,fnRowCallback:null,fnServerData:null,fnServerParams:null,fnStateLoadCallback:function(a){try{return JSON.parse((-1===a.iStateDuration?sessionStorage:localStorage).getItem("DataTables_"+a.sInstance+"_"+location.pathname))}catch(b){}},fnStateLoadParams:null,fnStateLoaded:null,fnStateSaveCallback:function(a,b){try{(-1=== fnFormatNumber:function(a){return a.toString().replace(/\B(?=(\d{3})+(?!\d))/g,this.oLanguage.sThousands)},fnHeaderCallback:null,fnInfoCallback:null,fnInitComplete:null,fnPreDrawCallback:null,fnRowCallback:null,fnServerData:null,fnServerParams:null,fnStateLoadCallback:function(a){try{return JSON.parse((-1===a.iStateDuration?sessionStorage:localStorage).getItem("DataTables_"+a.sInstance+"_"+location.pathname))}catch(b){}},fnStateLoadParams:null,fnStateLoaded:null,fnStateSaveCallback:function(a,b){try{(-1===
a.iStateDuration?sessionStorage:localStorage).setItem("DataTables_"+a.sInstance+"_"+location.pathname,JSON.stringify(b))}catch(c){}},fnStateSaveParams:null,iStateDuration:7200,iDeferLoading:null,iDisplayLength:10,iDisplayStart:0,iTabIndex:0,oClasses:{},oLanguage:{oAria:{sSortAscending:": activate to sort column ascending",sSortDescending:": activate to sort column descending"},oPaginate:{sFirst:"First",sLast:"Last",sNext:"Next",sPrevious:"Previous"},sEmptyTable:"No data available in table",sInfo:"Showing _START_ to _END_ of _TOTAL_ entries", a.iStateDuration?sessionStorage:localStorage).setItem("DataTables_"+a.sInstance+"_"+location.pathname,JSON.stringify(b))}catch(c){}},fnStateSaveParams:null,iStateDuration:7200,iDeferLoading:null,iDisplayLength:10,iDisplayStart:0,iTabIndex:0,oClasses:{},oLanguage:{oAria:{sSortAscending:": activate to sort column ascending",sSortDescending:": activate to sort column descending"},oPaginate:{sFirst:"First",sLast:"Last",sNext:"Next",sPrevious:"Previous"},sEmptyTable:"No data available in table",sInfo:"Showing _START_ to _END_ of _TOTAL_ entries",
sInfoEmpty:"Showing 0 to 0 of 0 entries",sInfoFiltered:"(filtered from _MAX_ total entries)",sInfoPostFix:"",sDecimal:"",sThousands:",",sLengthMenu:"Show _MENU_ entries",sLoadingRecords:"Loading...",sProcessing:"Processing...",sSearch:"Search:",sSearchPlaceholder:"",sUrl:"",sZeroRecords:"No matching records found"},oSearch:h.extend({},m.models.oSearch),sAjaxDataProp:"data",sAjaxSource:null,sDom:"lfrtip",searchDelay:null,sPaginationType:"simple_numbers",sScrollX:"",sScrollXInner:"",sScrollY:"",sServerMethod:"GET", sInfoEmpty:"Showing 0 to 0 of 0 entries",sInfoFiltered:"(filtered from _MAX_ total entries)",sInfoPostFix:"",sDecimal:"",sThousands:",",sLengthMenu:"Show _MENU_ entries",sLoadingRecords:"Loading...",sProcessing:"Processing...",sSearch:"Search Metadata:",sSearchPlaceholder:"",sUrl:"",sZeroRecords:"No matching records found"},oSearch:h.extend({},m.models.oSearch),sAjaxDataProp:"data",sAjaxSource:null,sDom:"lfrtip",searchDelay:null,sPaginationType:"simple_numbers",sScrollX:"",sScrollXInner:"",sScrollY:"",sServerMethod:"GET",
renderer:null,rowId:"DT_RowId"};X(m.defaults);m.defaults.column={aDataSort:null,iDataSort:-1,asSorting:["asc","desc"],bSearchable:!0,bSortable:!0,bVisible:!0,fnCreatedCell:null,mData:null,mRender:null,sCellType:"td",sClass:"",sContentPadding:"",sDefaultContent:null,sName:"",sSortDataType:"std",sTitle:null,sType:null,sWidth:null};X(m.defaults.column);m.models.oSettings={oFeatures:{bAutoWidth:null,bDeferRender:null,bFilter:null,bInfo:null,bLengthChange:null,bPaginate:null,bProcessing:null,bServerSide:null, renderer:null,rowId:"DT_RowId"};X(m.defaults);m.defaults.column={aDataSort:null,iDataSort:-1,asSorting:["asc","desc"],bSearchable:!0,bSortable:!0,bVisible:!0,fnCreatedCell:null,mData:null,mRender:null,sCellType:"td",sClass:"",sContentPadding:"",sDefaultContent:null,sName:"",sSortDataType:"std",sTitle:null,sType:null,sWidth:null};X(m.defaults.column);m.models.oSettings={oFeatures:{bAutoWidth:null,bDeferRender:null,bFilter:null,bInfo:null,bLengthChange:null,bPaginate:null,bProcessing:null,bServerSide:null,
bSort:null,bSortMulti:null,bSortClasses:null,bStateSave:null},oScroll:{bCollapse:null,iBarWidth:0,sX:null,sXInner:null,sY:null},oLanguage:{fnInfoCallback:null},oBrowser:{bScrollOversize:!1,bScrollbarLeft:!1,bBounding:!1,barWidth:0},ajax:null,aanFeatures:[],aoData:[],aiDisplay:[],aiDisplayMaster:[],aIds:{},aoColumns:[],aoHeader:[],aoFooter:[],oPreviousSearch:{},aoPreSearchCols:[],aaSorting:null,aaSortingFixed:[],asStripeClasses:null,asDestroyStripes:[],sDestroyWidth:0,aoRowCallback:[],aoHeaderCallback:[], bSort:null,bSortMulti:null,bSortClasses:null,bStateSave:null},oScroll:{bCollapse:null,iBarWidth:0,sX:null,sXInner:null,sY:null},oLanguage:{fnInfoCallback:null},oBrowser:{bScrollOversize:!1,bScrollbarLeft:!1,bBounding:!1,barWidth:0},ajax:null,aanFeatures:[],aoData:[],aiDisplay:[],aiDisplayMaster:[],aIds:{},aoColumns:[],aoHeader:[],aoFooter:[],oPreviousSearch:{},aoPreSearchCols:[],aaSorting:null,aaSortingFixed:[],asStripeClasses:null,asDestroyStripes:[],sDestroyWidth:0,aoRowCallback:[],aoHeaderCallback:[],
aoFooterCallback:[],aoDrawCallback:[],aoRowCreatedCallback:[],aoPreDrawCallback:[],aoInitComplete:[],aoStateSaveParams:[],aoStateLoadParams:[],aoStateLoaded:[],sTableId:"",nTable:null,nTHead:null,nTFoot:null,nTBody:null,nTableWrapper:null,bDeferLoading:!1,bInitialised:!1,aoOpenRows:[],sDom:null,searchDelay:null,sPaginationType:"two_button",iStateDuration:0,aoStateSave:[],aoStateLoad:[],oSavedState:null,oLoadedState:null,sAjaxSource:null,sAjaxDataProp:null,bAjaxDataGet:!0,jqXHR:null,json:k,oAjaxData:k, aoFooterCallback:[],aoDrawCallback:[],aoRowCreatedCallback:[],aoPreDrawCallback:[],aoInitComplete:[],aoStateSaveParams:[],aoStateLoadParams:[],aoStateLoaded:[],sTableId:"",nTable:null,nTHead:null,nTFoot:null,nTBody:null,nTableWrapper:null,bDeferLoading:!1,bInitialised:!1,aoOpenRows:[],sDom:null,searchDelay:null,sPaginationType:"two_button",iStateDuration:0,aoStateSave:[],aoStateLoad:[],oSavedState:null,oLoadedState:null,sAjaxSource:null,sAjaxDataProp:null,bAjaxDataGet:!0,jqXHR:null,json:k,oAjaxData:k,

View File

@@ -123,15 +123,6 @@ function load_gdoc_table(){
lengthMenu: [50,100,250,500] lengthMenu: [50,100,250,500]
}); });
// Get the search filter section and search box
var searchsec = $(filtlabel).find('label');
var searchbox = searchsec.find('input');
// Replace search filter section text,
// then re-add the removed search box
searchsec.text('Search Metadata: ');
searchsec.append(searchbox);
initGdocTable = true initGdocTable = true
}); });
console.log('Finished loading Google Drive master list'); console.log('Finished loading Google Drive master list');
@@ -190,15 +181,6 @@ function load_issue_table(){
lengthMenu: [50,100,250,500] lengthMenu: [50,100,250,500]
}); });
// Get the search filter section and search box
var searchsec = $(filtlabel).find('label');
var searchbox = searchsec.find('input');
// Replace search filter section text,
// then re-add the removed search box
searchsec.text('Search Metadata: ');
searchsec.append(searchbox);
initIssuesTable = true; initIssuesTable = true;
}); });
console.log('Finished loading Github issues master list'); console.log('Finished loading Github issues master list');
@@ -253,15 +235,6 @@ function load_ghfile_table(){
lengthMenu: [50,100,250,500] lengthMenu: [50,100,250,500]
}); });
// Get the search filter section and search box
var searchsec = $(filtlabel).find('label');
var searchbox = searchsec.find('input');
// Replace search filter section text,
// then re-add the removed search box
searchsec.text('Search Metadata: ');
searchsec.append(searchbox);
initGhfilesTable = true; initGhfilesTable = true;
}); });
console.log('Finished loading Github file list'); console.log('Finished loading Github file list');
@@ -314,15 +287,6 @@ function load_markdown_table(){
lengthMenu: [50,100,250,500] lengthMenu: [50,100,250,500]
}); });
// Get the search filter section and search box
var searchsec = $(filtlabel).find('label');
var searchbox = searchsec.find('input');
// Replace search filter section text,
// then re-add the removed search box
searchsec.text('Search Metadata: ');
searchsec.append(searchbox);
initMarkdownTable = true; initMarkdownTable = true;
}); });
console.log('Finished loading Markdown list'); console.log('Finished loading Markdown list');
@@ -374,15 +338,6 @@ function load_emailthreads_table(){
lengthMenu: [50,100,250,500] lengthMenu: [50,100,250,500]
}); });
// Get the search filter section and search box
var searchsec = $(filtlabel).find('label');
var searchbox = searchsec.find('input');
// Replace search filter section text,
// then re-add the removed search box
searchsec.text('Search Metadata: ');
searchsec.append(searchbox);
initEmailthreadsTable = true; initEmailthreadsTable = true;
}); });
console.log('Finished loading Groups.io email threads list'); console.log('Finished loading Groups.io email threads list');

View File

@@ -1,3 +1,61 @@
#modal-too-long {
visibility: hidden;
}
/* feedback smileys */
#modal-feedback-smile-icon,
#modal-feedback-frown-icon {
padding-left: 100px;
padding-right: 100px;
padding-top: 20px;
padding-bottom: 20px;
}
div.smile-active {
background-color: #2b2;
}
i.smile-active {
color: #fff;
}
div.frown-active {
background-color: #b22;
}
i.frown-active {
color: #fff;
}
/* feedback text area */
#modal-feedback-textarea {
width: 100%;
}
/* feedback buttons */
button.close {
font-size: 35px;
}
button#submit-feedback-btn {
width: 250px;
}
button#feedback:hover {
opacity: 1.0;
filter: alpha(opacity=100); /* For IE8 and earlier */
}
button#feedback {
opacity: 0.5;
filter: alpha(opacity=50); /* For IE8 and earlier */
width: 180px;
height: 50px;
position: fixed;
z-index: 999;
right: 120px;
bottom: 10px;
}
/* search results table */
td#search-results-score-col, td#search-results-score-col,
td#search-results-type-col { td#search-results-type-col {
width: 100px; width: 100px;

View File

@@ -1,4 +1,5 @@
{% extends "layout.html" %} {% extends "layout.html" %}
{% set active_page = "403" %}
{% block body %} {% block body %}
<div class="container"> <div class="container">

View File

@@ -1,4 +1,5 @@
{% extends "layout.html" %} {% extends "layout.html" %}
{% set active_page = "404" %}
{% block body %} {% block body %}
<div class="container"> <div class="container">

25
templates/banner.html Normal file
View File

@@ -0,0 +1,25 @@
<div class="container" id="banner-container">
{#
banner image
#}
<div class="row" id="banner-row">
<div class="col12sm" id="banner-col">
<center>
<a id="banner-a" href="{{ url_for('search')}}?query=&fields=">
<img id="banner-img" src="{{ url_for('static', filename='centillion_white.png') }}">
</a>
</center>
</div>
</div>
{% if config['TAGLINE'] %}
<div class="row" id="tagline-row">
<div class="col12sm" id="tagline-col">
<center>
<h2 id="tagline-tagline"> {{config['TAGLINE']}} </h2>
</center>
</div>
</div>
{% endif %}
</div>

View File

@@ -1,4 +1,5 @@
{% extends "layout.html" %} {% extends "layout.html" %}
{% set active_page = "control_panel" %}
{% block body %} {% block body %}
<hr /> <hr />

View File

@@ -0,0 +1,14 @@
<div id="messages">
{% with messages = get_flashed_messages() %}
{% if messages %}
<div class="container" id="flashed-messages-container">
<div class="alert alert-success alert-dismissible fade in">
<a href="#" class="close" data-dismiss="alert" aria-label="close">&times;</a>
{% for message in messages %}
<p class="lead">{{ message }}</p>
{% endfor %}
</div>
</div>
{% endif %}
{% endwith %}
</div>

View File

@@ -1,4 +1,5 @@
{% extends "layout.html" %} {% extends "layout.html" %}
{% set active_page = "landing" %}
{% block body %} {% block body %}
<div class="container"> <div class="container">
<div class="row"> <div class="row">

View File

@@ -10,6 +10,7 @@
<script src="{{ url_for('static', filename='bootstrap.min.js') }}"></script> <script src="{{ url_for('static', filename='bootstrap.min.js') }}"></script>
<script src="{{ url_for('static', filename='master_list.js') }}"></script> <script src="{{ url_for('static', filename='master_list.js') }}"></script>
<script src="{{ url_for('static', filename='search_list.js') }}"></script> <script src="{{ url_for('static', filename='search_list.js') }}"></script>
<script src="{{ url_for('static', filename='feedback.js') }}"></script>
{# ########## dataTables plugin ############ #} {# ########## dataTables plugin ############ #}
@@ -23,52 +24,43 @@
{# ########## github fork corner ############ #} <div id="master-div">
<div> {#
flashed messages
{% with messages = get_flashed_messages() %} #}
{% if messages %} {% include "flashed_messages.html" %}
<div class="container">
<div class="alert alert-success alert-dismissible">
<a href="#" class="close" data-dismiss="alert" aria-label="close">&times;</a>
<ul class=flashes>
{% for message in messages %}
<li>{{ message }}</li>
{% endfor %}
</ul>
</div>
</div>
{% endif %}
{% endwith %}
<div class="container">
{# {#
banner image banner image
#} #}
<div class="row"> {% include "banner.html" %}
<div class="col12sm">
<center>
<a href="{{ url_for('search')}}?query=&fields=">
<img src="{{ url_for('static', filename='centillion_white.png') }}">
</a>
{#
need a tag line
#}
{% if config['TAGLINE'] %}
<h2><a href="{{ url_for('search')}}?query=&fields=">
{{config['TAGLINE']}}
</a></h2>
{% endif %}
</center>
</div>
</div>
</div> {#
feedback modal
#}
{% include "modal.html" %}
{% block body %}{% endblock %} {% block body %}{% endblock %}
</div> </div>
<a href="https://github.com/dcppc/centillion" class="github-corner" aria-label="View source on Github"><svg width="80" height="80" viewBox="0 0 250 250" style="fill:#151513; color:#fff; position: absolute; top: 0; border: 0; right: 0;" aria-hidden="true"><path d="M0,0 L115,115 L130,115 L142,142 L250,250 L250,0 Z"></path><path d="M128.3,109.0 C113.8,99.7 119.0,89.6 119.0,89.6 C122.0,82.7 120.5,78.6 120.5,78.6 C119.2,72.0 123.4,76.3 123.4,76.3 C127.3,80.9 125.5,87.3 125.5,87.3 C122.9,97.6 130.6,101.9 134.4,103.2" fill="currentColor" style="transform-origin: 130px 106px;" class="octo-arm"></path><path d="M115.0,115.0 C114.9,115.1 118.7,116.5 119.8,115.4 L133.7,101.6 C136.9,99.2 139.9,98.4 142.2,98.6 C133.8,88.0 127.5,74.4 143.8,58.0 C148.5,53.4 154.0,51.2 159.7,51.0 C160.3,49.4 163.2,43.6 171.4,40.1 C171.4,40.1 176.1,42.5 178.8,56.2 C183.1,58.6 187.2,61.8 190.9,65.4 C194.5,69.0 197.7,73.2 200.1,77.6 C213.8,80.2 216.3,84.9 216.3,84.9 C212.7,93.1 206.9,96.0 205.4,96.6 C205.1,102.4 203.0,107.8 198.3,112.5 C181.9,128.9 168.3,122.5 157.7,114.1 C157.9,116.9 156.7,120.9 152.7,124.9 L141.0,136.5 C139.8,137.7 141.6,141.9 141.8,141.8 Z" fill="currentColor" class="octo-body"></path></svg></a><style>.github-corner:hover .octo-arm{animation:octocat-wave 560ms ease-in-out}@keyframes octocat-wave{0%,100%{transform:rotate(0)}20%,60%{transform:rotate(-25deg)}40%,80%{transform:rotate(10deg)}}@media (max-width:500px){.github-corner:hover .octo-arm{animation:none}.github-corner .octo-arm{animation:octocat-wave 560ms ease-in-out}}</style> {% if active_page=="search" or active_page=="master_list" %}
{# feedback button #}
<button id="feedback" type="button"
data-toggle="modal"
data-target="#myModal"
class="btn btn-lg">Send Feedback</button>
{# vertical spacing before the bottom, b/c of button #}
<div id="footer-whitespace" class="container">
<p>&nbsp;</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
</div>
{% endif %}
<a id="github-corner" href="https://github.com/dcppc/centillion" class="github-corner" aria-label="View source on Github"><svg width="80" height="80" viewBox="0 0 250 250" style="fill:#151513; color:#fff; position: absolute; top: 0; border: 0; right: 0;" aria-hidden="true"><path d="M0,0 L115,115 L130,115 L142,142 L250,250 L250,0 Z"></path><path d="M128.3,109.0 C113.8,99.7 119.0,89.6 119.0,89.6 C122.0,82.7 120.5,78.6 120.5,78.6 C119.2,72.0 123.4,76.3 123.4,76.3 C127.3,80.9 125.5,87.3 125.5,87.3 C122.9,97.6 130.6,101.9 134.4,103.2" fill="currentColor" style="transform-origin: 130px 106px;" class="octo-arm"></path><path d="M115.0,115.0 C114.9,115.1 118.7,116.5 119.8,115.4 L133.7,101.6 C136.9,99.2 139.9,98.4 142.2,98.6 C133.8,88.0 127.5,74.4 143.8,58.0 C148.5,53.4 154.0,51.2 159.7,51.0 C160.3,49.4 163.2,43.6 171.4,40.1 C171.4,40.1 176.1,42.5 178.8,56.2 C183.1,58.6 187.2,61.8 190.9,65.4 C194.5,69.0 197.7,73.2 200.1,77.6 C213.8,80.2 216.3,84.9 216.3,84.9 C212.7,93.1 206.9,96.0 205.4,96.6 C205.1,102.4 203.0,107.8 198.3,112.5 C181.9,128.9 168.3,122.5 157.7,114.1 C157.9,116.9 156.7,120.9 152.7,124.9 L141.0,136.5 C139.8,137.7 141.6,141.9 141.8,141.8 Z" fill="currentColor" class="octo-body"></path></svg></a><style>.github-corner:hover .octo-arm{animation:octocat-wave 560ms ease-in-out}@keyframes octocat-wave{0%,100%{transform:rotate(0)}20%,60%{transform:rotate(-25deg)}40%,80%{transform:rotate(10deg)}}@media (max-width:500px){.github-corner:hover .octo-arm{animation:none}.github-corner .octo-arm{animation:octocat-wave 560ms ease-in-out}}</style>

View File

@@ -1,4 +1,5 @@
{% extends "layout.html" %} {% extends "layout.html" %}
{% set active_page = "master_list" %}
{% block body %} {% block body %}
<hr /> <hr />

51
templates/modal.html Normal file
View File

@@ -0,0 +1,51 @@
<div class="modal fade" id="myModal" tabindex="-1" role="dialog" aria-labelledby="myModalLabel">
<form id="myModalForm" method="post">
<div class="modal-dialog" role="document">
<div id="myModal-content" class="modal-content">
<div id="myModal-header" class="modal-header">
<button type="button" class="close" data-dismiss="modal" aria-label="Close">
<span aria-hidden="true">&times;</span>
</button>
<h4 class="modal-title" id="myModalLabel">
Send us feedback!
</h4>
</div>
<div id="myModal-body" class="modal-body">
<div id="modal-feedback-smile-frown-container" class="container-fluid">
<div id="modal-feedback-smile-div" class="col-xs-6 text-center"
onClick="smile()">
<i id="modal-feedback-smile-icon" class="fa fa-smile-o fa-4x" aria-hidden="true"></i>
</div>
<div id="modal-feedback-frown-div" class="col-xs-6 text-center"
onClick="frown()">
<i id="modal-feedback-frown-icon" class="fa fa-frown-o fa-4x" aria-hidden="true"></i>
</div>
</div>
<div class="container-fluid">
<p>&nbsp;</p>
</div>
<div id="modal-feedback-textarea-container" class="container-fluid">
<textarea id="modal-feedback-textarea" rows="6"></textarea>
</div>
<div id="modal-too-long" class="container-fluid" >
<p id="modal-too-long-text" class="lead">Please limit the length of your feedback. Thank you in advance!</p>
</div>
</div>
<div id="myModal-footer" class="modal-footer">
<div class="text-center">
<button id="submit-feedback-btn" type="button"
onClick="submit_feedback()"
class="btn btn-lg btn-primary">
Send
</button>
</div>
</div>
</div>
</div>
</form>
</div>

View File

@@ -1,7 +1,8 @@
{% extends "layout.html" %} {% extends "layout.html" %}
{% set active_page = "search" %}
{% block body %} {% block body %}
<div class="container"> <div id="search-bar-container" class="container">
<div class="row"> <div class="row">
<div class="col-xs-12"> <div class="col-xs-12">
@@ -12,7 +13,11 @@
<p><button id="the-big-one" type="submit" style="font-size: 20px; padding: 10px; padding-left: 50px; padding-right: 50px;" <p><button id="the-big-one" type="submit" style="font-size: 20px; padding: 10px; padding-left: 50px; padding-right: 50px;"
value="search" class="btn btn-primary">Search</button> value="search" class="btn btn-primary">Search</button>
</p> </p>
{% if parsed_query %}
<p><a href="{{ url_for('search')}}?query=&fields=">[clear all results]</a> <p><a href="{{ url_for('search')}}?query=&fields=">[clear all results]</a>
{% endif %}
</p> </p>
</form> </form>
</center> </center>
@@ -20,17 +25,10 @@
</div> </div>
</div> </div>
<div class="container"> <div style="height: 20px;"><p>&nbsp;</p></div>
<div class="row">
{% if directories %} <div id="info-bars-container" class="container">
<div class="col-xs-12 info directories-cloud"> <div class="row">
<b>File directories:</b>
{% for d in directories %}
<a href="{{url_for('search')}}?query={{d|trim}}&fields=filename">{{d|trim}}</a>
{% endfor %}
</div>
{% endif %}
<ul class="list-group"> <ul class="list-group">
@@ -46,6 +44,9 @@
</li> </li>
{% endif %} {% endif %}
{# use "if parsed_query" to check if this is
a new search or search results #}
{% if parsed_query %} {% if parsed_query %}
<li class="list-group-item"> <li class="list-group-item">
<div class="container-fluid"> <div class="container-fluid">
@@ -59,6 +60,7 @@
</li> </li>
{% endif %} {% endif %}
<li class="list-group-item"> <li class="list-group-item">
<div class="container-fluid"> <div class="container-fluid">
<div class="row"> <div class="row">
@@ -99,7 +101,7 @@
</div> </div>
{% if parsed_query %} {% if parsed_query %}
<div class="container"> <div id="search-results-container" class="container">
<div class="row"> <div class="row">
<table id="search-results" class="table"> <table id="search-results" class="table">
<thead id="search-results-header"> <thead id="search-results-header">
@@ -126,44 +128,21 @@
{% if e.kind=="gdoc" %} {% if e.kind=="gdoc" %}
{% if e.mimetype=="document" %} {% if e.mimetype=="document" %}
<p><small>Drive Document</small</p> <p><small>Drive Document</small</p>
<!--
<i class="fa fa-google fa-2x"></i>
<i class="fa fa-file-text fa-2x"></i>
-->
{% else %} {% else %}
<p><small>Drive File</small</p> <p><small>Drive File</small</p>
<!--
<i class="fa fa-google fa-2x"></i>
<i class="fa fa-file-o fa-2x"></i>
-->
{% endif %} {% endif %}
{% elif e.kind=="issue" %} {% elif e.kind=="issue" %}
<p><small>Issue</small</p> <p><small>Issue</small</p>
<!--
<i class="fa fa-github fa-2x"></i>
<i class="fa fa-question fa-2x"></i>
-->
{% elif e.kind=="ghfile" %} {% elif e.kind=="ghfile" %}
<p><small>Github File</small</p> <p><small>Github File</small</p>
<!--
<i class="fa fa-github fa-2x"></i>
<i class="fa fa-file-o fa-2x"></i>
-->
{% elif e.kind=="markdown" %} {% elif e.kind=="markdown" %}
<p><small>Github Markdown</small</p> <p><small>Github Markdown</small</p>
<!--
<i class="fa fa-github fa-2x"></i>
<i class="fa fa-file-text-o fa-2x"></i>
-->
{% elif e.kind=="emailthread" %} {% elif e.kind=="emailthread" %}
<p><small>Email Thread</small</p> <p><small>Email Thread</small</p>
<!--
<i class="fa fa-envelope-o fa-2x"></i>
-->
{% else %} {% else %}
<p><small>Unknown</small</p> <p><small>Unknown</small</p>