Compare commits

...

4 Commits

5 changed files with 200 additions and 141 deletions

View File

@@ -67,7 +67,7 @@ List of topics focusing on theoretical components:
* Nonparametric Methods
* Decision Trees
* Lienar iscrimination
* Linear discrimination
* Miultilyaer Perceptrons
* Local Models
* Kernel Machines
@@ -76,7 +76,6 @@ List of topics focusing on theoretical components:
* Graphical Models
* Combining Multiple Learners
* Reinforcement Learning
* Design and Analysis of Machine Learning Experiments
@@ -84,8 +83,7 @@ List of topics focusing on theoretical components:
A long and full list of types of models under each sub-heading:
* Regression
* **Modeling relationship between variables, iteratively refined using an error measure.**
* Regression: **Modeling relationship between variables, iteratively refined using an error measure.**
* Linear Regression
* Logistic Regression
* OLS (Ordinary Least Squares) Regression
@@ -93,55 +91,47 @@ A long and full list of types of models under each sub-heading:
* MARS (Multivariate Adaptive Regression Splines)
* LOESS (Locally Estimated Scatterplot Smoothing)
* Instance Based
* **Build up database of data, compare new data to database; winner-take-all or memory-based learning.**
* Instance Based: **Build up database of data, compare new data to database; winner-take-all or memory-based learning.**
* k-Nearest Neighbor
* Learning Vector quantization
* Self-Organizing Map
* Localy Weighted Learning
* Regularization
* **Extension made to other methods, penalizes model complexity, favors simpler and more generalizable models.**
* Regularization: **Extension made to other methods, penalizes model complexity, favors simpler and more generalizable models.**
* Ridge Regression
* LASSO (Least Absolute Shrinkage and Selection Operator)
* Elastic Net
* LARS (Least Angle Regression)
* Decision Tree
* **Construct a model of decisions made on actual values of attributes in the data.**
* Decision Tree: **Construct a model of decisions made on actual values of attributes in the data.**
* Classification and Regression Tree
* CHAID (Chi-Squared Automatic Interaction Detection)
* Conditional Decision Trees
* Bayesian
* **Methods explicitly applying Bayes' Theorem for classification and regression problems.**
* Bayesian: **Methods explicitly applying Bayes' Theorem for classification and regression problems.**
* Naive Bayes
* Gaussian Naive Bayes
* Multinomial Naive Bayes
* Bayesian Network
* BBN (Bayesian Belief Network)
* Clustering
* **Centroid-based and hierarchical modeling approaches; groups of maximum commonality.**
* Clustering: **Centroid-based and hierarchical modeling approaches; groups of maximum commonality.**
* k-Means
* k-Medians
* Expectation Maximization
* Hierarchical Clustering
* Association Rule Algorithms
* **Extract rules that best explain relationships between variables in data.**
* Association Rule Algorithms: **Extract rules that best explain relationships between variables in data.**
* Apriori algorithm
* Eclat algorithm
* Neural Networks
* **Inspired by structure and function of biological neural networks, used ofr regression and classification problems.**
* Neural Networks: **Inspired by structure and function of biological neural networks, used ofr regression and classification problems.**
* Radial Basis Function Network (RBFN)
* Perceptron
* Back-Propagation
* Hopfield Network
* Deep Learning
* **Neural networks that exploit cheap and abundant computational power; semi-supervised, lots of data.**
* Deep Learning: **Neural networks that exploit cheap and abundant computational power; semi-supervised, lots of data.**
* Convolutional Neural Network (CNN)
* Recurrent Neural Network (RNN)
* Long-Short-Term Memory Network (LSTM)
@@ -149,8 +139,7 @@ A long and full list of types of models under each sub-heading:
* Deep Belief Network (DBN)
* Stacked Auto-Encoders
* Dimensionality Reduction
* **Find inherent structure in data, in an unsupervised manner, to describe data using less information.**
* Dimensionality Reduction: **Find inherent structure in data, in an unsupervised manner, to describe data using less information.**
* PCA
* t-SNE
* PLS (Partial Least Squares Regression)
@@ -164,8 +153,7 @@ A long and full list of types of models under each sub-heading:
* Regularized Discriminant Analysis
* Linear Discriminant Analysis
* Ensemble
* **Models composed of multiple weaker models, independently trained, that provide a combined prediction.**
* Ensemble: **Models composed of multiple weaker models, independently trained, that provide a combined prediction.**
* Random Forest
* Gradient Boosting Machines (GBM)
* Boosting
@@ -174,4 +162,3 @@ A long and full list of types of models under each sub-heading:
* Stacked Generalization (Blending)
* Gradient Boosted Regression Trees

View File

@@ -15,22 +15,13 @@ See [ThePlan.md](/ThePlan.md)
### Software Engineering
The basic concepts, the full topic list, the to do list.
Full list of software engineering topics: [SoftwareEngineering.md](/SoftwareEngineering.md)
See [SoftwareEngineering.md](/SoftwareEngineering.md)
### TODO Software Engineering
See [TODOSoftwareEngineering.md](/TODOSoftwareEngineering.md)
Schedule and task list (ended August 2017): [TODOSoftwareEngineering.md](/TODOSoftwareEngineering.md)
--------
### Machine Learning
The basic concepts and the detailed topic list.
Full list of machine learning topics: [MachineLearning.md](/MachineLearning.md)
See [MachineLearning.md](/MachineLearning.md)
### TODO Machine Learning
See [TODOMachineLearning.md](/TODOMachineLearning.md)

View File

@@ -9,9 +9,11 @@
* Parabolas
* Points, spaces, curves, 2D vs 3D
* Solutions of linear algebraic equations
* Linear system, matrix solvers
* Interpolation and extrapolation
* Integration
@@ -36,28 +38,21 @@
* Chapter 2 - solutions of linear algebraic equations
* Chapter 3 - interpolation and extrapolation
* Chapter 4 - Integration of Functions
* Chapter 5 - Evaluation of Functions
* Chapter 6 - Special Functions
Chapter 7 - Random numbers
Chapter 8 - Sorting and selection
Chapter 9 - Root finding and nonlinear equations
Chapter 10 - Minimization and maximization
Chapter 11 - Eigensystems
Chapter 12 - Fast Fourier Transform
Chapter 13 - Fourier and spectral applications
Chapter 14 - Statistical description of data
Chapter 15 - Modeling of data
Chapter 16 - Classification and Inference
Chapter 17 - Integration of ODEs
Chapter 18 - Two-point boundary solution
Chapter 19 - Integral equations and inverse theory
Chapter 20 - PDEs
Chapter 21 - Computational geometry
* Chapter 7 - Random numbers
* Chapter 8 - Sorting and selection
* Chapter 9 - Root finding and nonlinear equations
* Chapter 10 - Minimization and maximization
* Chapter 11 - Eigensystems
* Chapter 12 - Fast Fourier Transform
* Chapter 13 - Fourier and spectral applications
* Chapter 14 - Statistical description of data
* Chapter 15 - Modeling of data
* Chapter 16 - Classification and Inference
* Chapter 17 - Integration of ODEs
* Chapter 18 - Two-point boundary solution
* Chapter 19 - Integral equations and inverse theory
* Chapter 20 - PDEs
* Chapter 21 - Computational geometry

View File

@@ -1,7 +1,7 @@
# TODO List - Software Engineering
Table of Contents:
* [Data Structures](#Data Structures)
* [Data Structures](#data-structures)
* [Arrays](#arrays)
* [Linked Lists](#ll)
* [Stacks and Queues](#sq)
@@ -10,13 +10,9 @@ Table of Contents:
* [Dictionaries and Maps and Hash Tables](#dict)
* [Sets, Skip Lists](#sets)
* [Search Trees](#searchtrees)
* [Advanced Data Structures (Skiena)](#adv)
* [Graphs](#graphs)
* [Advanced Data Structures (Skiena)](#adv)
* [Study Guides](#studyguides)
* [Data Structures and Libraries](#lib)
* Guava
* Apache Commons
* Others
* [Algorithms](#algorithms)
* [Complexity and Big O Notation](#bigO)
* [Timing](#timing)
@@ -29,16 +25,17 @@ Table of Contents:
* [OOP](#oop)
* [Java](#java)
* [Practice and Applications](#practice)
* [Blogging and Wrtiting](#blogging)
* [Blogging and Writing](#blogging)
* [Project Euler](#euler)
* [ICPC](#icpc)
* [Google Code Jam](#codejam)
* [Engineering Interview Notes](#interview-notes)
<a name="data_structures"></a>
<a name="data-structures"></a>
## Data Structures
<a name="arrays"></a>
@@ -82,7 +79,6 @@ Table of Contents:
- Fisher Yates [https://charlesreid1.com/wiki/Arrays/Java/FisherYates](https://charlesreid1.com/wiki/Arrays/Java/FisherYates)
- Python list [https://charlesreid1.com/wiki/Arrays/Java/PythonList](https://charlesreid1.com/wiki/Arrays/Java/PythonList)
- Removing elements
- Code:
- Python-style list in Java: [https://charlesreid1.com:3000/cs/java/src/master/arrays/python-list/PythonList.java](https://charlesreid1.com:3000/cs/java/src/master/arrays/python-list/PythonList.java)
- Usage of the Array data type: [https://charlesreid1.com:3000/cs/java/src/master/arrays](https://charlesreid1.com:3000/cs/java/src/master/arrays)
@@ -112,7 +108,7 @@ Table of Contents:
---
[ ] Linked lists: Python
[X] Linked lists: Python
- Singly liked lists: 06/04
- Books:
- Goodrich Python Chapter 7
@@ -121,7 +117,6 @@ Table of Contents:
- [https://charlesreid1.com/wiki/Linked_Lists](https://charlesreid1.com/wiki/Linked_Lists)
- [https://charlesreid1.com/wiki/Linked_Lists/Python](https://charlesreid1.com/wiki/Linked_Lists/Python)
- [https://charlesreid1.com/wiki/Linked_Lists/Python/Single](https://charlesreid1.com/wiki/Linked_Lists/Python/Single)
- End of chapter questions:
- Code:
@@ -139,8 +134,6 @@ Table of Contents:
- Wiki notes:
- [https://charlesreid1.com/wiki/StacksQueues](https://charlesreid1.com/wiki/StacksQueues)
- [https://charlesreid1.com/wiki/StacksQueues/Python](https://charlesreid1.com/wiki/StacksQueues/Python)
- End of chapter questions:
- (none)
- Code:
- Python: [https://charlesreid1.com:3000/cs/python/src/master/stacks-queues-deques/](https://charlesreid1.com:3000/cs/python/src/master/stacks-queues-deques/)
- Array-based stack: [https://charlesreid1.com:3000/cs/python/src/master/stacks-queues-deques/stacks/ArrayStack.py](https://charlesreid1.com:3000/cs/python/src/master/stacks-queues-deques/stacks/ArrayStack.py)
@@ -165,9 +158,6 @@ Table of Contents:
- [https://charlesreid1.com/wiki/StacksQueues/Java/LinkedStack](https://charlesreid1.com/wiki/StacksQueues/Java/LinkedStack)
- [https://charlesreid1.com/wiki/StacksQueues/Java/LinkedQueue](https://charlesreid1.com/wiki/StacksQueues/Java/LinkedQueue)
- [https://charlesreid1.com/wiki/StacksQueues/Java/LinkedDeque](https://charlesreid1.com/wiki/StacksQueues/Java/LinkedDeque)
- End of chapter questions:
- Goodrich
- To do
- Implementations/Code:
- Array based stack: [https://charlesreid1.com:3000/cs/java/src/master/stacks-queues-deques/stacks/ArrayStack.java](https://charlesreid1.com:3000/cs/java/src/master/stacks-queues-deques/stacks/ArrayStack.java)
- Array based queue: [https://charlesreid1.com:3000/cs/java/src/master/stacks-queues-deques/queues/ArrayQueue.java](https://charlesreid1.com:3000/cs/java/src/master/stacks-queues-deques/queues/ArrayQueue.java)
@@ -197,13 +187,12 @@ Table of Contents:
- Goodrich trees coverage includes patterns, abstract methods
- Wiki notes:
- [https://charlesreid1.com/wiki/Template:TreesFlag](https://charlesreid1.com/wiki/Template:TreesFlag)
- End of chapter questions:
---
[ ] Trees: Java
[X] Trees: Java
- 06/10 - 06/13, 06/15 - 06/16
- Books:
- Goodrich Java Chapter 8
@@ -219,13 +208,14 @@ Table of Contents:
- Skiena: [https://charlesreid1.com/wiki/Tree/LogN_Min_Search](https://charlesreid1.com/wiki/Tree/LogN_Min_Search)
- Expression trees: [https://charlesreid1.com/wiki/Expression_Trees](https://charlesreid1.com/wiki/Expression_Trees)
- Another, more lightweight implementation of trees
- Important thing is not to have the fanciest implementation, but to have one that is fast and that works for your application.
- The important thing is not to have the fanciest implementation, but to have one that is fast and that works for your application.
- Example: implementing a stack node alone, rather than a full stack object, made it really fast to add and remove things.
- Sorted trees:
- Not covered in the Goodrich book...??
- See Skiena for coverage...
- Weiss C++ book also has good coverage.
- Sorted trees means balanced trees, and balanced trees introduces myriad strategies.
- Sorted trees means balanced trees
- Balanced trees introduces myriad strategies.
---
@@ -244,9 +234,9 @@ Table of Contents:
- Timing Notes:
- Spent a lot of time wrestling with the sorted and unsorted implementations. The unsorted was *way* slower than expected.
- Not sure what's going on. Need to go through and implement them according to the book.
- Finally got classes worked out, 6/22. Better-documented process. Documentation could still use some improvement. Consolidation still an issue.
- Finally got classes worked out, 6/22.
- Better-documented process. Documentation could still use some improvement. Consolidation still an issue.
- Package? Thinking like you are building an API. Maven.
- End of chapter questions:
- Git Code:
- Non-tree sorted/unsorted priority queues: use a built-in list data type
- [https://charlesreid1.com:3000/cs/java/src/master/priority-queues/AbstractPriorityQueue.java](https://charlesreid1.com:3000/cs/java/src/master/priority-queues/AbstractPriorityQueue.java)
@@ -257,17 +247,23 @@ Table of Contents:
---
[ ] Trees: heap trees
[X] Trees: heap trees
- 06/23
- Books:
- Goodrich Python/Java books covering priority queue theory/concepts
- Online videos:
- [Heaps and heap sort (MIT 6.006)](https://www.youtube.com/watch?v=B7hVxCmfPtM)
- Wiki notes:
- Binary search trees
- Heap sort
- Java Priority queues: [https://charlesreid1.com/wiki/Priority_Queues/Java](https://charlesreid1.com/wiki/Priority_Queues/Java)
- Priority queue ADT: [https://charlesreid1.com/wiki/Priority_Queues/ADT](https://charlesreid1.com/wiki/Priority_Queues/ADT)
- Priority queue (stored in sorted order): [https://charlesreid1.com/wiki/Priority_Queues/Sorted](https://charlesreid1.com/wiki/Priority_Queues/Sorted)
- Priority queue (stored in unsorted order): [https://charlesreid1.com/wiki/Priority_Queues/Unsorted](https://charlesreid1.com/wiki/Priority_Queues/Unsorted)
- Heaps: [https://charlesreid1.com/wiki/Priority_Queues/Heap](https://charlesreid1.com/wiki/Priority_Queues/Heap)
- Heaps with Java: [https://charlesreid1.com/wiki/Priority_Queues/Java](https://charlesreid1.com/wiki/Priority_Queues/Java)
- Custom comparators with priority queues: [https://charlesreid1.com/wiki/Priority_Queues/Comparators](https://charlesreid1.com/wiki/Priority_Queues/Comparators)
- End of chapter questions:
- Git Code
- max oriented priority queue: [https://charlesreid1.com/wiki/Maximum_Oriented_Priority_Queue](https://charlesreid1.com/wiki/Maximum_Oriented_Priority_Queue)
- stack implementation of pq: [https://charlesreid1.com/wiki/Priority_Queues/Stack](https://charlesreid1.com/wiki/Priority_Queues/Stack)
@@ -345,7 +341,7 @@ Table of Contents:
- [B-Trees insertion and sort](https://www.youtube.com/watch?v=k5J9M5_IMzg)
- Wiki notes:
- Binary search trees [https://charlesreid1.com/wiki/Binary_Search_Trees](https://charlesreid1.com/wiki/Binary_Search_Trees)
- Binary search tree navigation [
- Binary search tree navigation
- Binary search tree insertion and deletion
- OOP: Hooks, virtual method alternatives, positions vs nodes, key-value pairs
- AVL search tree
@@ -359,12 +355,55 @@ Table of Contents:
<a name="graphs"></a>
### Graphs
[ ] Graphs: directed and undirected
[ ] Graphs
- Books:
- Goodrich Chapter 14
- Wiki notes:
- Graph theory:
- Definitions: [https://charlesreid1.com/wiki/Graphs/Definitions](https://charlesreid1.com/wiki/Graphs/Definitions)
- Matching: [https://charlesreid1.com/wiki/Graphs/Matching](https://charlesreid1.com/wiki/Graphs/Matching)
- Connectivity: [https://charlesreid1.com/wiki/Graphs/Connectivity](https://charlesreid1.com/wiki/Graphs/Connectivity)
- First theorem of graph theory: [https://charlesreid1.com/wiki/First_Theorem_of_Graph_Theory](https://charlesreid1.com/wiki/First_Theorem_of_Graph_Theory)
- Implementations:
- Data structures: [https://charlesreid1.com/wiki/Graphs/Data_Structures](https://charlesreid1.com/wiki/Graphs/Data_Structures)
- Adjacency matrix:
- Adjacency list:
- Adjacency map: [https://charlesreid1.com/wiki/Graphs/Java/Adjacency_Map](https://charlesreid1.com/wiki/Graphs/Java/Adjacency_Map)
- Edge list:
- Guava: [https://charlesreid1.com/wiki/Graphs/Guava](https://charlesreid1.com/wiki/Graphs/Guava)
- Algorithms:
- Traversal: [https://charlesreid1.com/wiki/Graphs/Traversal](https://charlesreid1.com/wiki/Graphs/Traversal)
- BFS: [https://charlesreid1.com/wiki/Graphs/Breadth_First_Traversal](https://charlesreid1.com/wiki/Graphs/Breadth_First_Traversal)
- DFS: [https://charlesreid1.com/wiki/Graphs/Depth_First_Traversal](https://charlesreid1.com/wiki/Graphs/Depth_First_Traversal)
- Finding cycles: [https://charlesreid1.com/wiki/Graphs/Finding_Cycles](https://charlesreid1.com/wiki/Graphs/Finding_Cycles)
- Connected components: [https://charlesreid1.com/wiki/Graphs/Finding_Connected_Components](https://charlesreid1.com/wiki/Graphs/Finding_Connected_Components)
- Reachability: [https://charlesreid1.com/wiki/Graphs/Reachability](https://charlesreid1.com/wiki/Graphs/Reachability)
- Transitive closure: [https://charlesreid1.com/wiki/Graphs/Transitive_Closure](https://charlesreid1.com/wiki/Graphs/Transitive_Closure)
- Floyd Warshall: [https://charlesreid1.com/wiki/Graphs/Floyd_Warshall](https://charlesreid1.com/wiki/Graphs/Floyd_Warshall)
- Shortest path: [https://charlesreid1.com/wiki/Graphs/Shortest_Path](https://charlesreid1.com/wiki/Graphs/Shortest_Path)
- Edge relaxation: [https://charlesreid1.com/wiki/Graphs/Edge_Relaxation](https://charlesreid1.com/wiki/Graphs/Edge_Relaxation)
- Dijkstra: [https://charlesreid1.com/wiki/Graphs/Dijkstra](https://charlesreid1.com/wiki/Graphs/Dijkstra)
[ ] Graphs: Adjacency Matrix and List Representations of Graphs
[ ] Graphs: Traversals (BFS, DFS)
---
<a name="adv"></a>
### Skiena: Algorithms, Advanced Data Structures, Programming Challenges
[ ] Advanced Data Structures: Dictionaries, Priority Queues, Binary Trees, Hash Tables
- 06/02, 06/03
- Books:
- Skiena Chapter 3
- Wiki notes:
- [https://charlesreid1.com/wiki/Dictionaries](https://charlesreid1.com/wiki/Dictionaries)
- [https://charlesreid1.com/wiki/Priority_Queues](https://charlesreid1.com/wiki/Priority_Queues)
(This ended up being quite a jump in a different direction.
Much more practical focus on algorithms, only the most useful data structures,
only the core of each data structure, basic operations and big-O analysis.)
@@ -393,49 +432,17 @@ Table of Contents:
- Search trees study guide - finished 7/9
- [https://charlesreid1.com/wiki/Search_Trees_Study_Guide](https://charlesreid1.com/wiki/Search_Trees_Study_Guide)
[ ] Data structures flashcards
---
<a name="adv"></a>
### Skiena: Algorithms, Advanced Data Structures, Programming Challenges
[ ] Advanced Data Structures: Dictionaries, Priority Queues, Binary Trees, Hash Tables
- 06/02, 06/03
- Books:
- Skiena Chapter 3
- Wiki notes:
- [https://charlesreid1.com/wiki/Dictionaries](https://charlesreid1.com/wiki/Dictionaries)
- [https://charlesreid1.com/wiki/Priority_Queues](https://charlesreid1.com/wiki/Priority_Queues)
- End of chapter questions:
- Skiena chapter 3 question solutions on wiki
(This ended up being quite a jump in a different direction.
Much more practical focus on algorithms, only the most useful data structures,
only the core of each data structure, basic operations and big-O analysis.)
<a name="lib"></a>
## Data Structures Libraries
Some libraries that provide data structures in Java.
### Guava
[ ] Google Guava Library
---
### Apache Commons
[ ] Apache Commons Library
---
* Guava - [https://charlesreid1.com/wiki/Graphs/Guava](https://charlesreid1.com/wiki/Graphs/Guava)
* Apache Commons
@@ -611,21 +618,15 @@ Slower:
(For full list, see CS flag on wiki)
[ ] Interfaces, Abstract Classes, Virtual Methods
[ ] Interfaces
[ ] Testing (Unit testing)
[ ] Timing
[ ] Memory Usage
[ ] Garbage Collection
[ ] Lambda functions
[ ] Decorators
Java topics:
* Interfaces, Abstract Classes, Virtual Methods
* Interfaces
* Testing (Unit testing)
* Timing
* Memory Usage
* Garbage Collection
* Lambda functions
* Decorators
@@ -743,3 +744,91 @@ Solutions to Google Code Jam problems:
[https://charlesreid1.com:3000/charlesreid1/code-jam](https://charlesreid1.com:3000/charlesreid1/code-jam)
<a name="interview-notes"></a>
## Engineering Interview Notes
Link: [https://www.youtube.com/watch?v=ko-KkSmp-Lk](https://www.youtube.com/watch?v=ko-KkSmp-Lk)
Algorithms:
* Discuss space and time complexity
* System design and OOP
Resume:
* Be prepared to prove what is on your resume
Design questions:
* Work with your interviewer
Analysis questions:
* Ensure you understand all aspects of the problem
* Give multiple solutions
Code questions:
* Be able to come up with efficient code in a short amount of time
Coding:
* Be familiar with at least one coding language (C++ or Java)
* APIs
* OO design
* How to test
* Corner cases and edge cases
Algorithms:
* Time and space complexity
* How to improve or change
Data structures:
* As many as humanly possible
Testing:
* Unit testing
* Corner cases, edge cases
* End to end integration tests
* Security tests
NP Complete
* What does NP complete actually *mean*?
* TSP
* Knapsack problem
* Recognize these when they are presented in alternate forms
Math:
* Discrete math problems
* Counting, probability, combinatorics
Recursion:
* Practice
Operating systems:
* Processes, threads, concurrency issues
* Semaphores, mutexes, locks
* Resource allocation
* Context switching - an issue with the operating system and underlying hardware
* Scheduling
System design:
* Knowledge, theory, judgement for real world code
* Class hierarchies, distributed systems, design of systems under constraints
The Internet:
* Routers, domains, servers, load balancers, firewalls
* How search works
Tips:
* Explain and clarify - even if discarding ideas, vocalize the process
* Clarification: "Are there time or space complexity requirements?"
* Keep thinking through the problem - the first solution is not the best, so come up with more
* Practice physically writing your code
* Should be bug free

View File

@@ -58,8 +58,5 @@ Code:
- [C](https://git.charlesreid1.com/cs/c)
- [C++](https://git.charlesreid1.com/cs/cpp)
Practice writing out on a whiteboard and/or on paper,
before implementing on computer.
Get a big drawing pad from the art store.