diff --git a/AI/Artificial Intelligence/Ethics.md b/AI/Artificial Intelligence/Ethics.md new file mode 100644 index 0000000..7cc5cc6 --- /dev/null +++ b/AI/Artificial Intelligence/Ethics.md @@ -0,0 +1,67 @@ +--- +title: Ethics +updated: 2022-04-03 11:38:26Z +created: 2021-05-04 14:58:11Z +--- + +# Why is AI ethics becoming a problem now? + +Machine learning (ML) through neural networks is advancing rapidly for three reasons: + +1. Huge increase in the size of data sets; +2. Huge increase in computing power; +3. Huge improvement in ML algorithms and more human talent to write them. + +All three of these trends are centralizing of power, and “With great power comes great responsibility” + +## Sixteen Challenges + +1. **Technical Safety** + Will AI systems work as they are promised or will they fail? If and when they fail, what will be the results of those failures? And if we are dependent upon them, will we be able to survive without them? + Might legally reduce a manufacturer’s responsibility in terms and contracts => unethical scheme to avoid legitimate responsibility. +2. **Transparency and Privacy** + Once we have determined that the technology functions adequately, can we actually understand how it works and properly gather data on its functioning? Ethical analysis always depends on getting the facts first—only then can evaluation begin. + It turns out that with some machine learning techniques such as deep learning in neural networks it can be difficult or impossible to really understand why the machine is making the choices that it makes. In other cases, it might be that the machine can explain something, but the explanation is too complex for humans to understand. + As an additional point, in general, the more powerful someone or something is, the more transparent it ought to be, while the weaker someone is, the more right to privacy he or she should have. Therefore the idea that powerful AIs might be intrinsically opaque is disconcerting. +3. **Beneficial Use & Capacity for Good** + The main purpose of AI is to help people lead longer, more flourishing, more fulfilling lives. +4. **Malicious Use & Capacity for Evil** + Artificial intelligence, like human intelligence, will be used maliciously, there is no doubt. + Competition and war are always primary drivers of technological advance, and that militaries and corporations are working on these technologies right now. Not always intended. Because of this, forbidding, banning, and relinquishing certain types of technology would be the most prudent solution. +5. **Bias in Data, Training Sets, etc.** + One of the interesting things about neural networks is that they effectively merge a computer program with the data that is given to it. Many benefits but also potential harm. + Algorithmic bias is one of the major concerns in AI right now and will remain so in the future unless we endeavor to make our technological products better than we are. +6. **Unemployment / Lack of Purpose & Meaning** + Automation of industry has been a major contributing factor in job losses since the beginning of the industrial revolution. AI will simply extend this trend to more fields ie law, medicine, and education. + Attached to the concern for employment is the concern for how humanity spends its time and what makes a life well-spent. +7. **Growing Socio-Economic Inequality** + Related to the unemployment problem is the question of how people will survive if unemployment rises to very high levels. + Universal basic income (UBI) => major restructuring of national economies (political) +8. **Environmental Effects** + Machine learning models require enormous amounts of energy to train ==> fossil fuels. AI is in some very basic ways a technology focused on efficiency, and energy efficiency is one way that its capabilities can be directed. +9. **Automating Ethics** + One strength of AI is that it can automate decision-making. Automation of decision making will presents huge problems for society, because if these automated decisions are good, society will benefit, but if they are bad, society will be harmed. => ethical standards needed. + the ethical decision-making process might be as simple as following a program to fairly distribute a benefit, wherein the decision is made by humans and executed by algorithms, but it also might entail much more detailed ethical analysis, even if we humans would prefer that it did not—this is because Ai will operate so much faster than humans can, that under some circumstances humans will be left “out of the loop” of control due to human slowness. This already occurs with cyberattacks, and high-frequency trading (both of which are filled with ethical questions which are typically ignored) and it will only get worse as AI expands its role in society. + Since AI can be so powerful, the ethical standards we give to it had better be good. +10. **Moral Deskilling & Debility** + If we turn over our decision-making capacities to machines, we will become less experienced at making decisions. AI will be to either assist or replace humans at making certain types of decisions => humans may become worse at these skills. +11. **AI Consciousness, Personhood, and “Robot Rights”** + Some thinkers have wondered whether AIs might eventually become self-conscious, attain their own volition, or otherwise deserve recognition as persons like ourselves. + Legally => personhood has been given to corporations and (in other countries) rivers, so there is certainly no need for consciousness even before legal questions may arise. + Morally => perhaps someday they will be such good imitations. +12. **AGI and Superintelligence** + when AI reaches human levels of intelligence (AGI=Artificial General Intelligenc), potentially vastly more clever and capable than we are. + There is no reason why the improvement of AI would stop at AGI => more hardware to do more and faster. Dethroning of humanity as the most intelligent thing on Earth. +13. **Dependency on AI** + Our technological dependency is almost what defines us as a species. => complex and fragile + Intelligence dependence is a form of dependence like that of a child to an adult. raises the question of what an infantilized human race would do if our AI parents ever malfunctioned. Without that AI, if dependent on it, we could become like lost children not knowing how to take care of ourselves or our technological society. +14. **AI-powered Addiction** + AI can exploit numerous human desires and weaknesses including purpose-seeking, gambling, greed, libido, violence, and so on. Addiction enslaves us and wastes our time when we could be doing something worthwhile. It is not the AIs that choose to treat people this way, it is other people. +15. **Isolation and Loneliness** + One might think that “social” media, smartphones, and AI could help, but in fact they are major causes of loneliness since people are facing screens instead of each other. What does help are strong in-person relationships, precisely the relationships that are being pushed out by addictive (often AI-powered) technology. Loneliness can be helped by dropping devices and building quality in-person relationships. In other words: caring. +16. **Effects on the Human Spirit** + By externalizing our intelligence and improving it beyond human intelligence, are we making ourselves second-class beings to our own creations? + +[Artificial Intelligence and Ethics: Sixteen Challenges and Opportunities](https://www.scu.edu/ethics/all-about-ethics/artificial-intelligence-and-ethics-sixteen-challenges-and-opportunities/) + + diff --git a/AI/FeatureEngineering/DataStreaming.md b/AI/FeatureEngineering/DataStreaming.md new file mode 100644 index 0000000..e1752f4 --- /dev/null +++ b/AI/FeatureEngineering/DataStreaming.md @@ -0,0 +1,155 @@ +--- +title: DataStreaming +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# **Data Streaming** + +Data streaming is optimal for time series and detecting patterns over time. +Things like traffic sensors, health sensors, transaction logs, and activity logs are all good candidates for data streaming. + +## Data Streaming Challenges +- Plan for scalability. +- Plan for data durability. +- Incorporate fault tolerance in both the storage and processing layers. + +## Data Streaming Tools (popular) +- [Amazon Kinesis Data Firehose](https://aws.amazon.com/kinesis/data-firehose/) Managed, scalable, cloud-based service +- [Apache Kafka](https://kafka.apache.org/) Distributed publish-subscribe messaging system,integrates applications and data streams +- [Apache Flink](https://flink.apache.org/) Distributed computation over data streams +- [Apache Storm](https://storm.apache.org/) High data velocity. + +# **Devoxx Apache Kafka Presentation of James Ward** + +Integration Complexity: +- No System of recods; unknown where the data came initially +- Sychronisation is hard +- Scaling ETL is hard; horizonally is best, but fails often +- processing is error-prone; parsing, missing data ... + +### integration architecture +Events not tables +Streams as ledger (records all events and can go back in time) +First-class partitioning (scala horizontally) + +### Why not messaging system? +- is ordening garanteed? +- Horizontal scaling +- push? Back pressure? Difference in speed ie one source is faster then an other. +The Reactive Streams specification defines a model for **back pressure**. The elements in the Streams are produced by the producer at one end, and the elements are consumed by the consumer at the other end. The most favorable condition is where the rate at which the elements are produced and consumed is the same. But, in certain situations, the elements are emitted at a higher rate than they are consumed by the consumer. This scenario leads to the growing backlog of unconsumed elements. The more the backlog grows, the faster the application fails. Is it possible to stop the failure? And if yes, how to stop this failure? We can certainly stop the application from failing. One of the ways to do this is to communicate with the source ... etc + +### **Kafka = Event Ledge + Distributed & Redundant** +![Kafka](images/kafka1.png) + +### **Kafka: Liniar scaling** (near network speed between nodes/brokers) + +Kafka Fundamentals: +![Kafka Fundamentals](images/kafka2.png) +- messaging system Semantics +- Clustering is core (scaling horizontal) +- Durabiliy & Ordering Guarantees (no events are lost and in right order) + +Use cases: +![Use Cases](images/kafka3.png) +- modern ETL/CDC + (*change data capture (CDC) is a set of software design patterns used to determine (and track) the data that has changed so that action can be taken using the changed data. Also, Change data capture (CDC) is an approach to data integration that is based on the identification, capture and delivery of the changes made to enterprise data sources.*) +- Data Pipelines +![Data Pipeline example](images/kafka4.png) +- Big Data Ingest + + +**Records/events** +![Records/events](images/kafka5.png) +- Record: key, value, timestamp +- Immutable +- Append only to the ledger +- Persisted to disk (all on disk) + AKA: a log + +## Producers & Consumers +![Producers and Consumers](images/kafka6.png) +* Broker = Node in the cluster +* Producer writes records to a brober +* Consumer reads records from a broker (is asking for records) +* Leader/follower for cluster distribution + +## Topcs & Partitions +![Topcs & Partitions](images/kafka7.png) +* Topic = Logical name with 1..n partitions +* Partitions are replicated +* Ordering is guaranteed for a partition + +# Offset +![Offset](images/kafka8.png) +Offset the way Kafka keeps track of the ordering +* Unique sequential ID (per partition) +* Consumers track offsets (give me my offset) +* Benetis: replay, different speed consumers, etc + +# producer offset +![Producer Offset](images/kafka9.png) +Send message to given partion +* Wrirtes are to the leader of a partition (allways) +* Partitioning can be done mannually or based on a key +* Replication Factor is Topic-based +* Auto-rebalancing is arranged by Kafka +* followers (hot standby) are there when leader nodes go down. A new leader will be selected from the followers +* whole cluster is using Zookeeper to keep track of all nodes + +# Consumer groups +![Consumer groups](images/kafka10.png) +* Logical name for 1 or more consumers +* message consumption is load balanced across all consumers in a group + +# Delivery Guarantees +* **Producers:** + * Async (No guarantee Kafka has recorded; best performance) + * Committed to Leader + * depends saturation (verzadeging) node + * depends diskspeed + * Comitted to Leader and Quorum (part of folowers) + * very sure not loosing data + * Depends latency between nodes; so network latency + +* **Consumer:** + * at-least-once (default) + * Kafka delivers a block of records requested by the consumer. Begin and end offset are commited. If sending fails only the start was committed and redelivers all messages again. + * Kafka delivers and waits till consumer commit delivery + * at-most-once Right in the beginning is commited all messages are recieved. If fails halfway is pitty. + * effectively-once (at least one delivery) + * exactly-once (maybe; very difficult/impossible) + +# Cool features of Kafka +* log compaction option (removes duplicates) Group-by based on key. + * trade-off: disk read-writes; background proces + * gapes in offset-id. Kafka handles this. +* disk not heap + * kafka uses disk cache +* pagecache to socket: move piece of memory directly though the network; just skip Kafka; usefull for replication +* Balanced partition & leaders (Zookeeper) +* Producer and Consumer Quotes: to avoid fully saturated nodes +* Heraku Kafka + +# Clients +* JVM is official +* most other platforms via community +* polling based on consumption side + + +# AKKA Streams +![AKKA](images/kafka11.png) +* Implementation of Reactive streams +* Source/Sink Stream programming +* back-pressure etc +* [kafka-adapter](https://github.com/akka/alpakka-kafka) +* [Code examples](https://github.com/jamesward/koober) + +Sources: +https://dzone.com/articles/what-is-data-streaming +https://www.youtube.com/watch?v=UEg40Te8pnE DEVOXX KAFKA +https://en.wikipedia.org/wiki/Change_data_capture + + + + diff --git a/AI/FeatureEngineering/ELTvsETL.md b/AI/FeatureEngineering/ELTvsETL.md new file mode 100644 index 0000000..f79f9e0 --- /dev/null +++ b/AI/FeatureEngineering/ELTvsETL.md @@ -0,0 +1,82 @@ +--- +title: ELTvsETL +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# **ETL** +- Extraction (from source to staging area) +- Transformation (reformatted for datawarehouse purpose) +- Loaded (from staging area into datawarehous) + +![ETL process](images/ETL.png) +## ETL == Pipeline approach + +Many tools available: DataStage, Informatica, or SQL Server Integration Services (SSIS) ==> mostly for transforming +All work in a similair way: read from source, perform changes, write to target +ETL steps can be performed multiple times for a particulair load. +Transformation step can add business logic +Also everything is done in one single step. + +An ETL process should have data flowing steadily through it. Risk of running out of memory and/or disk space. (sorting is a classic example - holds entire dataset - if already sorted then rightaway in datawarehouse) Should have the possibility of buffering in pipeline. + +Many ETL tools facilitate parallel execution == multiple pipelines. +ETL can be better performing, but needs more training and developement cost. +When ETL is used: +- source and target differ in data types +- volumes of data are small +- transformations are compute-intensive +- data is structured + + + + + +# **ELT** +- Extract +- Load +- Transform (so data is transformed **AFTER** loading) +Target system is performing the transformation. +![ELT process](images/ELT.png) +## ELT == NO transformation engine + +When ELT is used: +- source and target share same data types (ie DB2 source and target) +- large volumes +- target database is able to handle large data volumes +- data is unstructured + + +ETL or ELT?? Depends on priorities. + +ELT is requires a powerful system in place as target. More such systems available because of analytics. + +ie a perfect platform is Hadoop but needs carefully planning + +| | ETL | ELT | +| --------------- | ------------------------------------------ | --------------------------------------------- | +| **Maturity** | for 20 years avaliable | not as well adapted | +| | expertise | works not well with structured data | +| **Flexibility** | Older ETL not suited for unstructured data | can handle structured and unstructured data | +| | Remap data => reload all preveous data! | In target data more flexible | +| **Hardware** | mostly own engine hardware | takes cumpute power from existing hardware | +| | modern ETL tools run in cloud | | +| **Better for** | - structured data | - Unstructured data | +| | - Smaller volumes and complex | - large volumes and less complex computations | +| | - On-premise relational databases | - Cloud environment | +| | | - Data lake | + +## **Merge ETL and ELT approach** +Extract, Transform, Load, Transform + + +## **Tools** +1. https://www.getdbt.com/ +2. https://fivetran.com/ +3. https://www.stitchdata.com/ + + + + +Sources: +https://dzone.com/articles/etl-vs-elt-differences-explained diff --git a/AI/MachineLearning/ML_Landscape.md b/AI/MachineLearning/ML_Landscape.md new file mode 100644 index 0000000..6d6b1ba --- /dev/null +++ b/AI/MachineLearning/ML_Landscape.md @@ -0,0 +1,177 @@ +--- +title: ML_Landscape +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# ML Landscape + +## What is ML + +Science of programming computers so they can learn from data, instead of having explicite code rules + +Data examples are called **training sets**. Each training example is called a **training instance** +Performance is measured in **accuracy** + +ML is good in solving problems that either: + +1. problems are too complex for traditonal approaches +2. have no known algoritme/can help find an algoritme +3. fluctuating environments: ML can learn machines to adapt to new data +4. can help humans learn to get insight in complex and large amount of data => data mining + +| | Description | Example | +| :---| :---- | :---- | +| CNN | Convolutional Neural Netwerk | Image Classification +| |Segmantic Segmentation | Brain scans | +| NLP | Natural Language Processing | News articles classification| +| | | Text Summary | +| RNN | Recurring Neural Netwerk | News articles classification | +| NLU | Natural Language Understanding | Chatbot/personal assistant | +| SVM | Support Vector Machine | Forecasting | +| RL | Reinformcement Learning | | + +**Regression model**: + +- Linear +- Polynomial +- Random Forest +- when take past in account then RNN, CNN, Transformers + +dimensionality reduction: Simplify the data without loosing too much information. +Feature extraction: merge one feature in an other and both will represent an new feature. + +Anomaly detection: unusual credit card transactions to prevent fraud. +Novelty detection: detact new instances that look different from all training instances. + +Association rule learning: dig in large datasets and discover interesting reations between attributes. + +## Types of ML + +1. **Supervised Learning** + - Classification + - K-Nearest Neighbours + - Linear/Logistic regression (both predictor and their labels required) + - SVM + - Decision Tree + - Random Forests + - Neural Networks +2. **Unsupervised Learning** + - Clustering + - K-Means + - DBSCAN + - HCA (Hierarchical Cluster Analysis) + - Anomaly detection and novelty detection + - One-class SVM + - Isolation Forest + - Visualistation and dimensionality reduction + - PCS (Principal Component Analysis) + - Kernel PCA + - LLE (Locally Linear Embedding) + - t-SNE (t-Distributed Stochastic Neigbor Embedding) + - Association rule learning + - Apriori + - Eclat +3. **Semi-supervised Learning** + - partly labeled + - mostly combination of supervised and unsupervised learning + - DBN (Deep Believe Networks) + - RBM (Resticted Boltzmann Machines) +4. **Reinforcement Learning** + - Call an agent, observe the environment, select and perform actions, get reward or penalty. Learns by itself. + +System can wheter or not learn increamentally from stream of incomming data + +1. **Batch Learning** + - must train with all data available (offline learning) + - can take many hours to train and requires resources + - when extreme much data, can be impossible to train or limited resources + - new data then train system from scratch and (automatic) deploy +2. **Online learning** + - train system incrementially, by feeding sequential data. + - either individually or in mini-batches + - great when data arrives in an continous flow + - great approach when system needs to adapt quicky to new data. + - requires less resources + - when data does not fitt in memory => out-of-core learining + then online learning perfect approach. + - important parameter: learning rate + - High then adapt quickly to chancing data, but forgets also quickly. + - Low then learns slower, bet less sensitive to new data. + - Problem when feed with bad data => performance will decline. + Need to monitor system. + +ML Systems how they generalize. Needs to prform well on new data! + +1. **Instance-based learning** + - learn examples by heart, then generalize to new cases by using similarity measures to compare. +2. **Model-based learning** + - build model of examples and then use the model to make predictions + - Use model selection to select an appropiate model and fully specifying its architecture (incl tune parameters) + +Inference: make predictions on new data. + +## Main challenges of ML + +"Bad algorithme" and "bad data" + +1. Insufficient quantity of training data + - Not always easy and/or cheap to get extra training data + - More data is better. +2. Nonrepresentative Training data + - Crusial the data represents the case to generalize about. + For both instant-based en model-based learning + - If sample too small => sampling noise + - If large sample but sampling method is flawed => sampling bias + Sampling method: how data is collected +3. Poor-Quality data + - Errors, outliers, noise => clean up data + - Clearly outliers => discard them of try fix errors + - missing few features => whether ignore attribute or fill in values manually +4. Irrelevant Features + - Come up with a good set of features for training => feauture engineering: + - feature selection (select most useful) + - feature extraction (combine existing features to more useful one) + - creating new features by gathering new data +5. Overfitting the training data + Overfitting => model performs well on training dat, but does not generalize well. + Overfitting happens when model is too complex relative to the amount and noisiness of the training data. + Overfitting solutions: + - simplify model by selecting model with fewer parameters, reducing number of attributes, constraining model. + Constraining model => **regularization** result: will fit less the training data but generalises better to new data. + Amount of reqularization is controlled by hyperparameters. A hyperparameter is a parameter of the learning algorithme (not of model) + - gather more training data + - reduce the noise in the training data (fix errors, remove outliers) +6. Underfitting the training data + - Model is too simple to learn the structure of the data. + - Solutions: + - select more powerful model, with more parameters + - improve feature engineering tasks + - reduce contrains on the model (eg regularisation) + +## Testing and validation + +Training set and test set, relate to each other 80-20% to 99-1%; depending the absolute size total data set. +Error rate in test set => generalization error +Training error low, but generalization error high => overfitting + +### Hyperparameter tuning an model section + +Holdout validation: keep part of the training set (= validation/development set) to validate several candidata models and select the best. Work mostly very well, except when the validation set is too small. + Solution: cross-validation => using many small validation sets and validate each model per validation set.(drawback: trainingtime is multiplied by number of validation sets) + +### Data Mismatch + +Most important rule: validation set and test set must be as representative as possible of the data used in production +Training set => to train model +Test set => evaluate after happy with the dev set +Dev + test set have to come from the SAME distribution (random shuffle data) + +1. define dev set + metric. Quickly iterate + idea -> code -> experiment + + | Set | percentage of data | + | :--- | --: | + | training | 98 | + | dev | 1 | + | test |1 | diff --git a/AI/MachineLearning/Statistics.md b/AI/MachineLearning/Statistics.md new file mode 100644 index 0000000..eb95bf7 --- /dev/null +++ b/AI/MachineLearning/Statistics.md @@ -0,0 +1,87 @@ +--- +title: Statistics +updated: 2022-04-02 15:10:58Z +created: 2021-05-04 14:58:11Z +--- + +# Statistics + +## Data Types + +- **Categorical** + - **Nominal Variables** + Intrinsic order of the labels: + Country of birth (Argentina, England, Germany) + Postcode + Vehicle make (Citroen, Peugeot, ...) + - **Ordinal Variables** + Can be meaningfully ordered are called ordinal: + Student's grade in an exam (A, B, C or Fail) + Days of the week (Monday = 1 and Sunday = 7) +- **Numerical** + - **Discrete** + how many cards in a game? + integers + - **Continuous** + height of a room + floating point numbers + +What are proportions? +Is an aggregation of nominal data to provide a numerical figure. eq a percentage of nominal variables. + +## Mixed Variables + +- Observations show either numbers or categories among their values + - Number of credit accounts (1-100, U, T, M) U = unknown, T = unverified, M = unmatched) +- Observations show both numbers and categories in their values + - Cabin (Titanic) (A15, B18, ...) + + +## Distributions +![48751b057b60e03ec51f64e3235fa1b3.png](../../_resources/48751b057b60e03ec51f64e3235fa1b3.png) +Selecting something on de x-axis in the middle has an higher probability then rarer on to the edges. +Bell curve of Normal Distribution + +![be8b17237548f72ecd8013f80df036dc.png](../../_resources/be8b17237548f72ecd8013f80df036dc.png) +Bi-mode distribution + +![b7f8b2f785a9637ea5a22abe2877bca5.png](../../_resources/b7f8b2f785a9637ea5a22abe2877bca5.png) +Skewed distribution + +Sample Distribution +![34262aff59c5f5dd9a413b2b3d74629a.png](../../_resources/34262aff59c5f5dd9a413b2b3d74629a.png) + +$$ +\overline{X} == variance +$$ + + +## Sampling ande Estimation +eg +some best number of successes divided by the sampling, gives an estimate 10 / 3 = 3,3333 +$$ +\Theta == estimate with some variance around to make a good guess out of the sample +$$ +![846c953521751f708bd680556dc9ae0b.png](../../_resources/846c953521751f708bd680556dc9ae0b.png) +So given an sample we have am 95% confidence out sample estimate is in this interval. +If less sure of this theta, the larger the confidence interval. eq because the n is much smaller. + +![d575f021de579d10e3855c763198e7bc.png](../../_resources/d575f021de579d10e3855c763198e7bc.png) + +## Hypothesis Testing + +![981ea34418a595b422aab0b0df23f4b6.png](../../_resources/981ea34418a595b422aab0b0df23f4b6.png) +In Hypothesis Testing never: +- prove anything +- never accept the null hypothesis + +## P-values +consider a null Hypothesis: +Hypothesis Test asses ig our sample is extreme enough to reject the null. +The p-value then measure how extreme our sample is. + +![6af1399567c87fcda04a6414efbe18bf.png](../../_resources/6af1399567c87fcda04a6414efbe18bf.png) + + +## P-hacking +![ace369b638966681b9558c42e25dd0b4.png](../../_resources/ace369b638966681b9558c42e25dd0b4.png) \ No newline at end of file diff --git a/AI/Tensorflow/Installation.md b/AI/Tensorflow/Installation.md new file mode 100644 index 0000000..9f64bfc --- /dev/null +++ b/AI/Tensorflow/Installation.md @@ -0,0 +1,41 @@ +--- +title: Installation +updated: 2022-04-03 11:39:19Z +created: 2021-10-28 18:57:40Z +latitude: 52.38660000 +longitude: 5.27820000 +altitude: 0.0000 +--- + +# Installation (no need anymore, see conda install) + +[Installation cuda](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu-installation) +[Download page](https://developer.nvidia.com/cuda-downloads) +### Installation (possible different version) +wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin +sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600 +wget https://developer.download.nvidia.com/compute/cuda/11.5.0/local_installers/cuda-repo-ubuntu2004-11-5-local_11.5.0-495.29.05-1_amd64.deb +sudo dpkg -i cuda-repo-ubuntu2004-11-5-local_11.5.0-495.29.05-1_amd64.deb +sudo apt-key add /var/cuda-repo-ubuntu2004-11-5-local/7fa2af80.pub +sudo apt-get update +sudo apt-get -y install cuda + +sudo apt-get install cuda-compat-11-5 + +[Install cuddn](https://docs.nvidia.com/deeplearning/cudnn/archives/cudnn-824/install-guide/index.html#install-linux) +[Download page](https://developer.nvidia.com/rdp/cudnn-download) + +### Unzip the cuDNN package: +tar -xzvf cudnn-x.x-linux-x64-v8.x.x.x.tgz + +### Copy the following files into the CUDA Toolkit directory: +$ sudo cp cuda/include/cudnn*.h /usr/local/cuda/include +$ sudo cp -P cuda/lib64/libcudnn* /usr/local/cuda/lib64 +$ sudo chmod a+r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn* + + +[Post actions](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions) +# environment variables add to .bashrc +export LD_LIBRARY_PATH=/usr/local/cuda-11.5/lib64\${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} + +export PATH=/usr/local/cuda-11.5/bin${PATH:+:${PATH}} diff --git a/AI/Tensorflow/testingGPU.md b/AI/Tensorflow/testingGPU.md new file mode 100644 index 0000000..6aa56ac --- /dev/null +++ b/AI/Tensorflow/testingGPU.md @@ -0,0 +1,39 @@ +--- +title: testingGPU +updated: 2022-04-03 09:01:13Z +created: 2021-05-04 14:58:11Z +--- + +### list devices CPU, GPU + +```python +tf.config.experimental.list_physical_devices("GPU") +from tensorflow.python.client import device_lib +print(device_lib.list_local_devices()) +``` + +### Init GPU; disable experimentals +```python +physical_devices = tf.config.list_physical_devices('GPU') +tf.config.experimental.set_memory_growth(physical_devices[0], True) +tf.config.experimental.disable_mlir_graph_optimization() +tf.config.experimental.enable_tensor_float_32_execution(enabled=True) +``` + +### assign memory to GPU +```python +gpus = tf.config.experimental.list_physical_devices("GPU") + +if gpus: + # Restrict TensorFlow to only allocate 22GB of memory on the first GPU + try: + tf.config.experimental.set_virtual_device_configuration( + gpus[0], + [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=7000)], + ) + logical_gpus = tf.config.experimental.list_logical_devices("GPU") + print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs") + except RuntimeError as e: + # Virtual devices must be set before GPUs have been initialized + print(e) +``` diff --git a/AI/nvidea/nvidea.md b/AI/nvidea/nvidea.md new file mode 100644 index 0000000..afcc10b --- /dev/null +++ b/AI/nvidea/nvidea.md @@ -0,0 +1,70 @@ +--- +title: nvidea +updated: 2022-04-03 11:44:16Z +created: 2021-05-04 14:58:11Z +--- + +# NVIDIA + +## show installed video drivers + +nvidia-smi + +[Latest drivers](https://www.nvidia.com/Download/index.aspx?lang=en-us) + +--- + +## list installed hw + +lspci | grep -i nvidia +sudo lshw -numeric -C display + +## find NVIDIA modules + +find /usr/lib/modules -name nvidia.ko + +## Settings + +nvidia-settings + +## run +```bash +nvidia-smi nvidia-smi -L +nvidia-smi -l n # run every n seconds +``` + + +## monitoring nvidia +https://github.com/fbcotter/py3nvml + +--- + +## successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero => error; Modify in host and set the -1 to 0 +/sys/bus/pci/devices/0000:2b:00.0/numa_node + +for a in /sys/bus/pci/devices/*; do echo 0 | sudo tee -a $a/numa_node; done + +https://stackoverflow.com/questions/44232898/memoryerror-in-tensorflow-and-successful-numa-node-read-from-sysfs-had-negativ + +--- + +## set numa value at start computer + +```bash +sudo crontab -e +# Add the following line +@reboot (echo 0 | tee -a "/sys/bus/pci/devices//numa_node") +``` +[Source](https://askubuntu.com/questions/1379119/how-to-set-the-numa-node-for-an-nvidia-gpu-persistently) + +--- + +## start docker with --gpus=all every time, otherwise error +### failed call to cuInit: UNKNOWN ERROR (-1 +### no NVIDIA GPU device is present: /dev/nvidia0 does not exist +docker run -it -p 8888:8888 --gpus=all tensorflow/tensorflow:latest-gpu-jupyter + +--- + +## update nvidea drivers +ubuntu-drivers autoinstall \ No newline at end of file diff --git a/Apache/Apache Hive/Create table.md b/Apache/Apache Hive/Create table.md new file mode 100644 index 0000000..91cf31d --- /dev/null +++ b/Apache/Apache Hive/Create table.md @@ -0,0 +1,89 @@ +--- +title: Create table +updated: 2022-04-04 11:59:19Z +created: 2022-04-03 12:44:49Z +--- + +```hive +CREATE [TEMPORARY] TABLE employee ( +name STRING, +work_place ARRAY, +gender_age STRUCT, +skills_score MAP, +depart_title MAP> +) +ROW FORMAT DELIMITED +FIELDS TERMINATED BY '|' +COLLECTION ITEMS TERMINATED BY ',' +MAP KEYS TERMINATED BY ':' +STORED AS TEXTFILE; + +LOAD DATA INPATH '/user/aap/data/employee.txt' +OVERWRITE INTO TABLE employee; +``` + +```hive +CREATE TABLE IF NOT EXISTS employee_hr ( +name string, +employee_id int, +sin_number string, +start_date date +) +ROW FORMAT DELIMITED +FIELDS TERMINATED BY '|'; + +LOAD DATA INPATH '/user/aap/data/employee_hr.txt' +OVERWRITE INTO TABLE employee_hr +``` + +```hive +CREATE TABLE employee_id ( +name STRING, +employee_id INT, +work_place ARRAY, +gender_age STRUCT, +skills_score MAP, +depart_title MAP> +) +ROW FORMAT DELIMITED +FIELDS TERMINATED BY '|' +COLLECTION ITEMS TERMINATED BY ',' +MAP KEYS TERMINATED BY ':'; + +LOAD DATA INPATH +'/user/aap/data/employee_id.txt' +OVERWRITE INTO TABLE employee_id +``` + +```hive +CREATE TABLE IF NOT EXISTS employee_contract ( +name string, +dept_num int, +employee_id int, +salary int, +type string, +start_date date +) +ROW FORMAT DELIMITED +FIELDS TERMINATED BY '|' +STORED as TEXTFILE; + +LOAD DATA INPATH '/user/aap/data/employee_contract.txt' +OVERWRITE INTO TABLE employee_contract; +``` + +```hive +CREATE TABLE ctas_employee as SELECT * FROM employee +``` + +```hive +CREATE VIEW IF NOT EXISTS employee_skills +AS +SELECT +name, skills_score['DB'] as DB, +skills_score['Perl'] as Perl, +skills_score['Python'] as Python, +skills_score['Sales'] as Sales, +skills_score['HR'] as HR +FROM employee; +``` diff --git a/Apache/Apache Hive/Extracting queries from Hive logs.md b/Apache/Apache Hive/Extracting queries from Hive logs.md new file mode 100644 index 0000000..3f53201 --- /dev/null +++ b/Apache/Apache Hive/Extracting queries from Hive logs.md @@ -0,0 +1,7 @@ +--- +title: 'Extracting queries from Hive logs ' +updated: 2022-04-27 17:36:19Z +created: 2022-04-27 17:36:05Z +--- + +https://thisdataguy.com/2017/06/23/extracting-queries-from-hive-logs/ \ No newline at end of file diff --git a/Apache/Apache Hive/Hive.md b/Apache/Apache Hive/Hive.md new file mode 100644 index 0000000..e835788 --- /dev/null +++ b/Apache/Apache Hive/Hive.md @@ -0,0 +1,20 @@ +--- +title: Hive +updated: 2022-05-24 18:43:47Z +created: 2022-05-24 18:35:26Z +--- + +The Apache Hive ™ data warehouse software facilitates reading, writing, and managing large datasets residing in distributed storage using SQL. Structure can be projected onto data already in storage. A command line tool and JDBC driver are provided to connect users to Hive. + +Built on top of Apache Hadoop™, Hive provides the following features: + +- Tools to enable easy access to data via SQL, thus enabling data warehousing tasks such as extract/transform/load (ETL), reporting, and data analysis. +- A mechanism to impose structure on a variety of data formats +- Access to files stored either directly in **Apache HDFS™** or in other data storage systems such as **Apache HBase™** +- Query execution via **Apache Tez™, Apache Spark™**, or **MapReduce** +- Procedural language with HPL-SQL +- Sub-second query retrieval via Hive LLAP, Apache YARN and Apache Slider. + +Hive's SQL can also be extended with user code via user defined functions (**UDF**s), user defined aggregates (UDAFs), and user defined table functions (UDTFs). + +Hive is not designed for online transaction processing (OLTP) workloads. It is best used for traditional data warehousing tasks. \ No newline at end of file diff --git a/Apache/Apache Hive/_ Aggregations.md b/Apache/Apache Hive/_ Aggregations.md new file mode 100644 index 0000000..967eb20 --- /dev/null +++ b/Apache/Apache Hive/_ Aggregations.md @@ -0,0 +1,31 @@ +--- +title: '# Aggregations' +updated: 2022-04-03 17:09:07Z +created: 2022-04-03 17:00:47Z +--- + +```hive +SELECT +sum(CASE WHEN gender_age.gender = 'Male' THEN gender_age.age ELSE 0 END)/ +count(CASE WHEN gender_age.gender = 'Male' THEN 1 +ELSE NULL END) as male_age_avg +FROM employee; + +SELECT +sum(coalesce(gender_age.age,0)) as age_sum, +sum(if(gender_age.gender = 'Female',gender_age.age,0)) as female_age_sum +FROM employee; + +SELECT +if(name = 'Will', 1, 0) as name_group, +count(name) as name_cnt +FROM employee +GROUP BY if(name = 'Will', 1, 0); +``` + +```hive +SELECT +count(DISTINCT gender_age.gender) as gender_uni_cnt, +count(DISTINCT name) as name_uni_cnt +FROM employee; +``` diff --git a/Apache/Apache Hive/_ DDL.md b/Apache/Apache Hive/_ DDL.md new file mode 100644 index 0000000..669b090 --- /dev/null +++ b/Apache/Apache Hive/_ DDL.md @@ -0,0 +1,17 @@ +--- +title: '# DDL' +updated: 2022-04-04 20:03:49Z +created: 2022-04-03 13:46:28Z +--- + +```hive +SHOW CREATE TABLE employee + +SHOW TABLES +SHOW TABLES '*em*' +SHOW VIEWS + +SHOW COLUMNS IN employee +DESC employee | DESCRIBE employee +SHOW TBLPROPERTIES employee +``` diff --git a/Apache/Apache Hive/_ Data Sorting.md b/Apache/Apache Hive/_ Data Sorting.md new file mode 100644 index 0000000..c3a9557 --- /dev/null +++ b/Apache/Apache Hive/_ Data Sorting.md @@ -0,0 +1,41 @@ +--- +title: '# Data Sorting' +updated: 2022-04-03 16:53:16Z +created: 2022-04-03 16:50:40Z +--- + +- ORDER BY [ASC|DESC] +It performs a global sort using only one reducer, so it takes longer to return the result. Using LIMIT with ORDER BY is strongly recommended. + +```hive +SELECT name +FROM employee -- Order by expression +ORDER BY CASE WHEN name = 'Will' THEN 0 ELSE 1 END DESC; + +SELECT * FROM emp_simple +ORDER BY work_place NULL LAST; +``` + +- SORT BY [ASC|DESC]: which columns to use to sort reducer input records. This means the sorting is completed before sending data to the reducer. + +```hive +SELECT name FROM employee SORT BY name DESC; +``` + +- DISTRIBUTE BY: It is very similar to GROUP BY when the mapper decides to which reducer it can deliver the output. Compared to GROUP BY, DISTRIBUTE BY will not work on data aggregations, such as count(*), but only directs where data goes + +```hive +SELECT name, employee_id FROM employee_hr DISTRIBUTE BY employee_id; + +SELECT name, start_date +FROM employee_hr +DISTRIBUTE BY start_date SORT BY name; +``` + +- CLUSTER BY: shortcut operator you can use to perform DISTRIBUTE BY and SORT BY operations on the same group of columns. The CLUSTER BY statement does not allow you to specify ASC or DESC yet. Compared to ORDER BY, which is globally sorted, the CLUSTER BY statement sorts data in each distributed group: + +```hive +SELECT name, employee_id FROM employee_hr CLUSTER BY name; +``` + +![e9effef3a9891b908b2197d351856eff.png](../../_resources/e9effef3a9891b908b2197d351856eff.png) diff --git a/Apache/Apache Hive/_ Functions.md b/Apache/Apache Hive/_ Functions.md new file mode 100644 index 0000000..2328afc --- /dev/null +++ b/Apache/Apache Hive/_ Functions.md @@ -0,0 +1,43 @@ +--- +title: '# Functions' +updated: 2022-04-03 16:58:17Z +created: 2022-04-03 13:36:09Z +--- + +```hive +SELECT concat('1','+','3','=',cast((1 + 3) as string)) as res; + +SELECT +SIZE(work_place) as array_size, +SIZE(skills_score) as map_size, +SIZE(depart_title) as complex_size, +SIZE(depart_title["Product"]) as nest_size +FROM employee; + +SELECT +array_contains(work_place, 'Toronto') as is_Toronto, +sort_array(work_place) as sorted_array +FROM employee; +``` + +## Date + +```hive +SELECT TO_DATE(FROM_UNIXTIME(UNIX_TIMESTAMP())) as currentdate; +``` + +```hive +SELECT +reverse(split(reverse('/user/john/data/employee.txt'),'/')[0]) +as linux_file_name; +``` + +## Opposite of explode + +```hive +SELECT +collect_set(gender_age.gender) as gender_set, +collect_list(gender_age.gender) as gender_list +FROM employee; +``` + diff --git a/Apache/Apache Hive/_ Select.md b/Apache/Apache Hive/_ Select.md new file mode 100644 index 0000000..aa8fd8e --- /dev/null +++ b/Apache/Apache Hive/_ Select.md @@ -0,0 +1,108 @@ +--- +title: '# Select' +updated: 2022-04-07 19:29:20Z +created: 2022-04-03 13:25:15Z +--- + +```hive +select name, -- regular column +work_place[0], -- array +gender_age.gender, -- struct +skills_score['DB'], -- map +depart_title[0] -- map with array +from employee +``` + +```hive +select name, work_place, +cities +from employee +LATERAL VIEW explode(work_place) C AS cities; +``` + +```hive +select name, work_place, +depart_title['Product'], +jobs +from employee +LATERAL VIEW explode(depart_title['Product']) C AS jobs; +``` + +```hive +SELECT +name, +dept_num as deptno, +salary, +count(*) OVER (PARTITION BY dept_num) as cnt, +count(distinct dept_num) OVER (PARTITION BY dept_num) as dcnt, +sum(salary) OVER(PARTITION BY dept_num ORDER BY dept_num) as sum1, +sum(salary) OVER(ORDER BY dept_num) as sum2, +sum(salary) OVER(ORDER BY dept_num, name) as sum3 +FROM employee_contract +ORDER BY deptno, name; +``` + +```hive +with r1 as (select name from employee), +r2 as (select name from employee) +select * from r1 +union all +select * from r2 +``` + +```hive +SELECT +CASE WHEN gender_age.gender = 'Female' THEN 'Ms.' +ELSE 'Mr.' END as title, +name, +IF(array_contains(work_place, 'New York'), 'US', 'CA') as country +FROM employee; +``` + +```hive +SELECT +name, gender_age.gender as gender +FROM ( +SELECT * FROM employee WHERE gender_age.gender = 'Male' +) t1 -- t1 here is mandatory +``` + +```hive +SELECT name, gender_age FROM employee WHERE gender_age.age in (27, 30) +``` + +```hive +SELECT +name, gender_age +FROM employee +WHERE (gender_age.gender, gender_age.age) IN +(('Female', 27), ('Male', 27 + 3)) -- expression support version > v2.1.0 +``` + +|Join type | Logic | Rows returned | +|---|---|---| +|table_m JOIN table_n | This returns all rows matched in both tables.| m ∩ n| +|table_m LEFT JOIN table_n | This returns all rows in the left table and matched rows in the right table. If there is no match in the right table, it returns NULL in the right table.| m | +|table_m RIGHT JOIN table_n | This returns all rows in the right table and matched rows in the left table. If there is no match in the left table, it returns NULL in the left table.| n | +|table_m FULL JOIN table_n| This returns all rows in both tables and matched rows in both tables. If there is no match in the left or right table, it returns NULL instead. | m + n - m ∩ n | +|table_m CROSS JOIN table_n | This returns all row combinations in both the tables to produce a Cartesian product.| m * n | + +### Special joins for HiveQL + +- MAPJOIN: The MapJoin statement reads all the data from the small table to memory and broadcasts to all maps. During the map phase, the join operation is performed by comparing each row of data in the big table with small tables against the join conditions. Because there is no reduce needed, such kinds of join usually have better performance. In the newer version of Hive, Hive automatically converts join to MapJoin at runtime if possible. However, you can also manually specify the broadcast table by providing a join. +hint, /*+ MAPJOIN(table_name) */. The MapJoin operation does not support the following: Using MapJoin after UNION ALL, LATERAL VIEW, GROUP BY/JOIN/SORT BY/CLUSTER, and BY/DISTRIBUTE BY Using MapJoin before UNION, JOIN, and another MapJoin + +```hive +SELECT +/*+ MAPJOIN(employee) */ emp.name, emph.sin_number +FROM employee emp +CROSS JOIN employee_hr emph +WHERE emp.name <> emph.name; +``` + +- LEFT SEMI JOIN statement is also a type of MapJoin. It is the same as a subquery with IN/EXISTS after v0.13.0 of Hive. However, it is not recommended for use since it is not part of standard SQL + +```hive +SELECT a.name FROM employee a +LEFT SEMI JOIN employee_id b ON a.name = b.name; +``` diff --git a/Apache/Apache Spark/Cheat Sheat.md b/Apache/Apache Spark/Cheat Sheat.md new file mode 100644 index 0000000..2a65b24 --- /dev/null +++ b/Apache/Apache Spark/Cheat Sheat.md @@ -0,0 +1,189 @@ +--- +title: Cheat Sheat +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Cheat Sheat + +## Python, Spark setting + +```bash +# Spark home for full install +export SPARK_HOME="/usr/local/spark/" +# Set a fixed value for the hash seed secret +export PYTHONHASHSEED=0 +# Set an alternate Python executable +export PYSPARK_PYTHON=/usr/local/ipython/bin/ipython +# Augment the default search path for shared libraries +export LD_LIBRARY_PATH=/usr/local/ipython/bin/ipython +# Augment the default search path for private libraries +export PYTHONPATH=$SPARK_HOME/python/lib/py4j-*-src.zip:$PYTHONPATH:$SPARK_HOME/python/ +``` + +### Initializing SparkSession + +```python +from pyspark.sql import SparkSession +spark = SparkSession + .builder + .appName("Python Spark SQL basic example") + .config("spark.executor.memory", "1gb") + .config("spark.some.config.option", "some-value") + .getOrCreate + +sc = spark.sparkContext +``` + +### Creating DataFrames + +```python +from pyspark.sql.types import * + +# Infer Schema +sc = spark.sparkContext +lines = sc.textFile("people.txt") +parts = lines.map(lambda l: l.split(",")) +people = parts.map(lambda p: Row(name=p[0],age=int(p[1]))) +df_people = spark.createDataFrame(people) + +# Specify Schema +people = parts.map(lambda p: Row(name=p[0],age=int(p[1].strip()))) +schemaString = "name age" +fields = [StructField(field_name, StringType(), True) for field_name in schemaString.split()] +schema = StructType(fields) +spark.createDataFrame(people, schema).show() +``` + +### From Spark Data Sources + +```python +# JSON +df = spark.read.json("customer.json") +df2 = spark.read.load("people.json", format="json") + +# Parquet files +df3 = spark.read.load("users.parquet") + +# TXT files +df4 = spark.read.text("people.txt") +``` + + | Inspect Data | Inspect Data | + | ------------ | --------------------- | + | df.types | df.describe().show() | + | df.show() | df.columns | + | df.head() | df.count() | + | df.first() | df.distinct().count() | + | df.take(2) | df.printSchema() | + | df.schema | df.explain() | + +### Duplicate Values + +```python +df = df.dropDuplicates() +``` + +### Queries + +```python +from pyspark.sql import functions as F + +# Select +df.select('firstName', + 'lastName', + explode('phoneNumber').alias('contactInfo'), + "adddress.type", # type of address column + df['age'] + 10 + ).show() + +# When +# Show firstName and 0 or 1 depending on age > 30 +df.select("firstName",F.when(df.age > 30, 1).otherwise(0)).show() +# Show firstName if in the given options +df[df.firstName.isin("Jane","Boris")].collect() +df1.withColumn("new column",when(df1["major"] == "J",1).otherwise(0)).show() + +# Like +df.select("firstName", df.lastName.like("Smith")).show() + +# Startswith - Endswith +df.select("firstName", df.lastName.startswith("Sm")).show() +df.select(df.lastName.endswith("th")).show() + +# Substring +df.select(df.firstName.subs(1,3).alias("name")) + +# Between +df.select(df.age.between(22, 24)) +``` + +### Add, Update, Remove Columns + +```python +# Adding Columns +from pyspark.sql.types import * +df = df.withColumn('city',df.address.city) + .withColumn('postalCode',df.address.postalCode) + .withColumn('state',df.address.state) .withColumn('streetAddress',df.address.streetAddress) + .withColumn('telePhoneNumber', explode(df.phoneNumber.number)) + .withColumn('telePhoneType', explode(df.phoneNumber.type)) + .withColumn("medianHouseValue", df["medianHouseValue"].cast(FloatType()) + +from pyspark.sql.functions import add_months,current_date, year, dayofmonth, when +df2.select(add_months(df2.dt, 1).alias('next_month')).collect() +df3 = df2.withColumn("day",dayofmonth(current_date())) +df3.withColumn("year",when(year(current_date()) < 2020,year(current_date())).otherwise(2020)).show() + +# Updating Column ame +df = df.withColumnRenamed('telePhoneNumber', 'phoneNumber') + +# Removing Columns +df = df.drop("address", "phoneNumber") +df = df.drop(df.address).drop(df.phoneNumber) + +# GroupBy +df.groupBy("age").count() + +# Filter +df.filter(df["age"]>24) + +# Sort +peopledf.sort(peopledf.age.desc()) +df.sort("age", ascending=False) +df.orderBy(["age","city"],ascending=[0,1]) + +# Missing & Replacing Values +df.na.fill(50) +df.na.drop() +df.na.replace(10,20) + +# Repartitioning +df.repartittion(10).rdd.getNumPartitions() # df with 10 partitions +df.coalesce(1).rdd.getNumPartitions() # df with 1 partions +``` + +### Running SQL Queries Programmatically + +```python +# Registering DataFrames & Query as Views +df.createOrReplaceTempView("customer") +df.createTempView("customer") +df5 = spark.sql("SELECT * FROM customer") + +peopledf.createGlobalTempView("people") +peopledf2 = spark.sql("SELECT * FROM global_temp.people") +``` + +### Output + +```python +# Data Structures +rdd1 = df.rdd +df.toJSON().first() +df.toPandas() + +# Write & Save to Files +df.select("firstName", "city").write.save('someName.parquet') +df.select("firstName", "age").write.save('someName.json',format='json') +``` diff --git a/Apache/Apache Spark/Configuarions.md b/Apache/Apache Spark/Configuarions.md new file mode 100644 index 0000000..b9fc962 --- /dev/null +++ b/Apache/Apache Spark/Configuarions.md @@ -0,0 +1,23 @@ +--- +title: Configuarions +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Spark configuation options + +SPARK_LOCAL_IP environment variable + +```bash +SPARK_LOCAL_IP=127.0.0.1 ./bin/spark-shell +``` + +In a program set the bindAddress + +```scala +val config = new SparkConf() +config.setMaster("local[*]") +config.setAppName("Test App") +config.set("spark.driver.bindAddress", "127.0.0.1") +val sc = new SparkContext(config) +``` diff --git a/Apache/Apache Spark/Files.md b/Apache/Apache Spark/Files.md new file mode 100644 index 0000000..375d98f --- /dev/null +++ b/Apache/Apache Spark/Files.md @@ -0,0 +1,26 @@ +--- +title: Files +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +Files + +## Get remote files + +```python +from pyspark import SparkContext +from pyspark import SparkFiles +from pyspark.sql import SQLContext + +url = "https://raw.githubusercontent.com/.../data/adult.csv" + +sc = SparkContext() +sc.addFile(url) +spark = SQLContext(sc) + +df = spark \ + .read \ + .csv(SparkFiles.get("adult.csv"),header=True,inferSchema=True) +df.printSchema() +``` diff --git a/Apache/Apache Spark/General.md b/Apache/Apache Spark/General.md new file mode 100644 index 0000000..0c43f3c --- /dev/null +++ b/Apache/Apache Spark/General.md @@ -0,0 +1,22 @@ +--- +title: General +updated: 2022-05-24 19:25:58Z +created: 2022-05-24 19:20:33Z +--- + +What is Apache Spark™? +Apache Spark™ is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters. + +### Batch/streaming data +> Unify the processing of your data in batches and real-time streaming, using your preferred language: Python, SQL, Scala, Java or R. + +### SQL analytics +> Execute fast, distributed ANSI SQL queries for dashboarding and ad-hoc reporting. Runs faster than most data warehouses. + +### Data science at scale +> Perform Exploratory Data Analysis (EDA) on petabyte-scale data without having to resort to downsampling + +### Machine learning +> Train machine learning algorithms on a laptop and use the same code to scale to fault-tolerant clusters of thousands of machines. + +[Difference between Spark DataFrame and Pandas DataFrame](https://www.geeksforgeeks.org/difference-between-spark-dataframe-and-pandas-dataframe/) \ No newline at end of file diff --git a/Apache/Apache Spark/Notes.md b/Apache/Apache Spark/Notes.md new file mode 100644 index 0000000..a2cbe01 --- /dev/null +++ b/Apache/Apache Spark/Notes.md @@ -0,0 +1,500 @@ +--- +title: Notes +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +Notes + +# Apache Spark Architecture + +Spark applications consists of + +- driver process + - maintaining information about the Spark Application; + - responding to a user’s program or input; + - analyzing, distributing, and scheduling work across the executors; +- executor processes + - executing code assigned to it by the driver; + - reporting the state of the computation on that executor back to the driver node; + +![Architecture Spark Application](https://raw.githubusercontent.com/jjmw/Summaries/master/images/architectureSparkApplication.png) + +## cluster manager + +controls physical machines and allocates +resources to Spark Applications: + +- Spark’s standalone cluster manager +- YARN +- Mesos + +Spark in _local mode_: driver and executor are simply processes. + +### Language APIs + +All languages (R, Python, Scala, Java) have similar performance characteristics when using structured API's. +When Python is using UDF then performance drops. Reason the Python code is executed in a separate Python process outside the JVM. + +Spark has two fundamental sets of APIs: + +- low-level "unstructured" (RDD) +- higher-level "structured" (Dataframe and Dataset) + +### Spark Session + +```scala +val myRange = spark.range(1000).toDF("number") +``` + +range of numbers represents a distributed collection: each part of this range of numbers exists on a different executor + +### Partitions + +To allow every executor to perform work in parallel, Spark breaks up the data into chunks called partitions. A partition is a collection of rows that sit on one physical machine in your cluster. A dataFrame’s partitions represent how the data is physically distributed across the cluster of machines during execution. If you have one partition, Spark will have a parallelism of only one, even if you have thousands of executors. If you have many partitions but only one executor, Spark will still have a parallelism of only one because there is only one computation resource + +```scala +spark.conf.set("spark.sql.shuffle.partitions", "5") +``` + +Default is 200 partitions + +### Transformations + + Core data structures are immutable. To “change” , ie a DataFrame, you need to instruct Spark how you would like to modify it to do what you want. These instructions are called *transformations*. Return no output (**lazy evaluation**) This is because we specified only an abstract transformation, and Spark will not act on transformations until we call an action, but build up a plan of transformations (predicate pushdown) + +Types of transformations: + +- narrow: one input partition ==> one output partition. In memory +- wide: input partitions ==> many output partitions (shuffle= Spark writes to disk) ie aggregation and sort +- lazy evaluation: Spark will wait until the very last moment to execute the graph of computation instructions. Spark compiles this plan from your raw DataFrame transformations to a streamlined physical plan that will run as efficiently aspossible across the cluster. + +![phtsicalPlan](../../_resources/5_physical_plan.png) + +### Actions + +An action instructs Spark to compute a result from a series of transformations. ie count. + +Kind of actions: + +- view data in the console +- collect data to native objects in the respective language +- write to output data sources + +![Architecture Spark Application](../../_resources/ReadSortTakeDataframe.png) + +### logical plan + +The logical plan of transformations that we build up defines a lineage for the DataFrame so that at any given point in time, Spark knows how to recompute any partition by performing all of the operations it had before on the same input data + +### DataFrames and SQL + +Register any DataFrame as a table or +view (a temporary table) and query it using pure SQL. There is **no performance** difference between writing SQL queries or writing DataFrame code, they both “compile” to the same +underlying plan that we specify in DataFrame code + +```scala +flightData2015.createOrReplaceTempView("flight_data_2015") + +val sqlWay = spark + .sql("""SELECT DEST_COUNTRY_NAME, count(1) FROM flight_data_2015 GROUP BY DEST_COUNTRY_NAME +""") + +val dataFrameWay = flightData2015 + .groupBy('DEST_COUNTRY_NAME) + .count() +``` + +--- + +## Spark Toolset + +![Spark toolset](../../_resources/spark_toolset.png) + +### Running Production Applications (spark-submit) + +spark-submit does one thing: it lets you send your application code to a cluster and launch it to execute there + +On local machine: + +```bash +## specify location of external jars +LIB=...... +JARS=$(files=("$LIB"/*.jar); IFS=,; echo "${files[*]}") + +./bin/spark-submit \ +--class org.apache.spark.examples.SparkPi \ +--master local \ +--jars $JARS \ +./examples/jars/spark-examples_2.11-2.2.0.jar 10 +``` + +### Datasets: Type-Safe Structured APIs + +- datasets: statically type code in Java and Scala. Is parameterized DataSet[T].In Scala use case classes. Collection of typed objects ie Scala Seq + +Reason to use Datasets: especially attractive for writing large applications, with which multiple software engineers must interact through well-defined interfaces. + +```scala +case class Flight(DEST_COUNTRY_NAME: String, +ORIGIN_COUNTRY_NAME: String, count: BigInt) + +val flightsDF = spark.read.parquet("/data/ +2010-summary.parquet/") + +val flights = flightsDF.as[Flight] +``` + +Advantage of Dataset use is that when call collect or take on a Dataset, it will collect objects of the proper type in your Dataset, not DataFrame Rows. This makes it easy to get type safety and securely perform manipulation in a distributed and a local manner without code changes + +### Dataframes +distibuted collection of objects of type Row + +### Structured Streaming + +Structured Streaming is a high-level API for stream processing that became production-ready in Spark 2.2. +It also makes it easy to conceptualize because you can write your batch job as a way to prototype it and then you can convert it to a streaming job + +### Machine Learning and Advanced Analytics + +Machine learning algorithms in MLlib require that data is represented as numerical values. All machine learning algorithms in Spark take as input a Vector type + +### Lower-Level APIs + +Virtually everything in Spark is built on +top of RDDs. One thing that you might use RDDs for is to parallelize raw data that you have stored in memory on the driver machine. +RDDs are available in Scala as well as Python. However, they’re not equivalent. + +### Spark’s Ecosystem and Packages + +[Spark Packages](https://spark-packages.org/) + +--- + +## Structured APIs + +The Structured APIs are the fundamental abstraction that you will use to write the majority of your data flows. +DataFrames and Datasets represent immutable, lazily evaluated plans that specify what operations to apply to data residing at a location to generate some output. +Support schema on write and schema on read. + +Spark uses an engine called **Catalys** and Spark is a program language in it own. The majority of our manipulations will operate strictly on Spark types. + +Within the Structured APIs, two more APIs: + +- untyped Dataframes; typed at runtime +- typed Datasets at compile time + +The “Row” type is Spark’s internal representation of its optimized in-memory format for computation. DataFrames are simply Datasets of Type Row. + +### Columns + +Columns represent a simple type like an integer or string a complex type like an array or map, or a null value. + +### Spark Types + +To work with the correct Scala types, use the following: + +```scala +import org.apache.spark.sql.types._ +val b = ByteType +``` + +Te get the Spark type: +the Scala type ie: + +- Short ==> ByteType +- Int ==> IntegerType +- etc + +### Overview of Structured API Execution + +single structured API query steps: + +1. Write DataFrame/Dataset/SQL Code +2. If valid code, Spark converts this to a Logical Plan. +3. Spark transforms this Logical Plan to a Physical Plan, checking for optimizations along the way +4. Spark then executes this Physical Plan (RDD manipulations) on the cluster. + +overview +![Catalyst Optimizer](../../_resources/CatalystOptimizer.png) + +Logical plan is first created and represents a set of abstract transformations that do not refer to executors or drivers. This plan is unresolved because although your code might be valid, the tables or columns that it refers to might or might not exist. Spark uses the catalog, a repository of all table and DataFrame information, to resolve columns and tables in the analyzer. The analyzer might reject the unresolved logical plan if the required table or column name does not exist in the catalog. Packages can extend the Catalyst to include their own rules for domain-specific optimizations. + +![LogicalPlan Spark](../../_resources/LogicalPlanSpark.png) + +The physical plan, often called a Spark plan, specifies how the logical plan will execute on the cluster by generating different physical execution strategies and comparing them through a cost model. Physical planning results in a series of RDDs and transformations + +![PhysicalPlan Spark](../../_resources/PhysicalPlanSpark.png) + +### Execution + +Upon selecting a physical plan, Spark runs all of this code over RDDs. further optimizations at runtime, generating native Java bytecode that can remove entire tasks or stages during execution. + +--- + +## Basic Structured Operations + +### Schemas + +A schema is a StructType made up of: + +- a number of fields +- StructFields, that have a name +- type (Spark types) +- Boolean flag: column can contain missing or null values +- optionally specify associated metadata with that column + +The metadata is a way of storing information about this column (Spark uses this in its machine learning library). + +```scala +import org.apache.spark.sql.types.{StructField,StructType, StringType,LongType} +import org.apache.spark.sql.types.Metadata + +val myManualSchema = StructType(Array( +StructField("DEST_COUNTRY_NAME", StringType, true), +StructField("ORIGIN_COUNTRY_NAME", StringType, true), +StructField("count", LongType, false, +Metadata.fromJson("{\"hello\":\"world\"}")) +)) + +val df = spark + .read + .format("json") + .schema(myManualSchema) + .load("/data/2015-summary.json") +``` + +### Columns and Expressions + +You cannot manipulate an individual column outside the context of a DataFrame; you must use Spark transformations +within a DataFrame to modify the contents of a column. + +Different ways to construct and refer to columns: + +```scala +import org.apache.spark.sql.functions.{col, column} +col("someColumnName") +column("someColumnName") +$"someColumnName" +'someColumnName + +df.col("count") // use +``` + +**Columns are not resolved until we compare the column names with those we are maintaining in the catalog. Column and table resolution happens in the analyzer +phase.** + +### Expressions + +Columns are expressions. An expression is a set of transformations on one or more values in a record in a DataFrame. (a function that takes as input one or more column names, resolves them, and then potentially +applies more expressions to create a single value for each record in the dataset) + + Each row in a DataFrame is a single record as an object of type Row + +### Creating Rows + +Only DataFrames have schemas. Rows themselves do not have +schemas. + +```scala +import org.apache.spark.sql.Row +val myRow = Row("Hello", null, 1, false) + +myRow(0) // type Any +myRow(0).asInstanceOf[String] // String +myRow.getString(0) // String +myRow.getInt(2) // Int +``` + +### Creating DataFrames + +```scala +val df = spark + .read + .format("json") + .load("/data/2015-summary.json") +df.createOrReplaceTempView("dfTable") +``` + +or + +```scala +import org.apache.spark.sql.Row +import org.apache.spark.sql.types.{StructField,StructType,StringType,LongType} + +val myManualSchema = new StructType(Array( +new StructField("some", StringType, true), +new StructField("col", StringType, true), +new StructField("names", LongType, false))) +val myRows = Seq(Row("Hello", null, 1L)) + +val myRDD = spark + .sparkContext + .parallelize(myRows) + +val myDf = spark + .createDataFrame(myRDD, myManualSchema) + +myDf.show() +``` + +### select and selectExpr + +```scala +mDF.select("colA","colB").show() +mDF.select('colA).show() +mDF.select(col("colA")).show() +mDF.select(expr("colA as aap")).show() // most flexible +mDF.select(expr("colA").alias("aap")).show() +mDF.selectExpr("colA as aap", "colB").show() // daily use; opens up the true power of Spark. +``` + +### Adding, renaming and dropping Columns + +Dataframe is NOT modified!! + +```scala +df.withColumn("numberOne", lit(1)) +df.withColumn("withinCountry", expr("ORIGIN_COUNTRY_NAME == DEST_COUNTRY_NAME")) + +df.withColumnRenamed("DEST_COUNTRY_NAME","dest") + +df.drop("DEST_COUNTRY_NAME") + +``` + +### Case Sensitivity + +By default Spark is case insensitive. Make sensitive: + +```scala +spark.sql("""set spark.sql.caseSensitive true""") +``` + +### Changing a Column’s Type (cast) + +```scala +df.withColumn("count2", col("count").cast("long")) +``` + +### Filtering Rows and Unique Rows + +```scala +df.filter(col("count") < 2).show(2) +df.where("count < 2").show(2) + +df.select("ORIGIN_COUNTRY_NAME", "DEST_COUNTRY_NAME") + .distinct().count() +``` + +### Random Samples and Split + +```scala +// Random Sample +val seed = 5 +val withReplacement = false +val fraction = 0.5 +df.sample(withReplacement, fraction, seed).count() + +// Random Splits +val dataFrames = df + .randomSplit(Array(0.25, 0.75), seed) +dataFrames(0) + .count() > dataFrames(1) + .count() // False +``` + +--- + +## Working with Different Types of Data + +**lit function**. This function converts a type in another language to its correspnding Spark representation. + +```scala +import org.apache.spark.sql.functions.lit +df.select(lit(5), lit("five"), lit(5.0)) +``` + +```scala +// where as in sql; === equal =!= not equal +vlucht.where(col("count") === 15) + .select("*") + .show(10) + +// best way +vlucht.where("count = 15").show(10) + +// column aap boolean; == equal +vlucht.selectExpr("*","count == 15 as aap").show(10) +``` + +**Better just write SQL!!!!** + +```scala +// compute summary statistics +df.describe().show() +``` + +### Working with Dates and Timestamps + +There are dates, which focus exclusively on calendar dates, and timestamps, which include both date and time information. Spark’s TimestampType class supports only second-level precision, which means that if you’re going to be working with milliseconds or microseconds, you’ll need to work around this problem by potentially operating on them as longs. + +### Working with Complex Types + +Structs: +Think of structs as DataFrames within DataFrames + +```scala +import org.apache.spark.sql.functions.struct +val complexDF = df.select(struct("Description","InvoiceNo") + .alias("complex")) + +complexDF.select(col("complex") + .getField("Description")) + .show() +``` + +split + +```scala +import org.apache.spark.sql.functions.split +df.select(split(col("Description"), " ") + .alias("array_col")) + .selectExpr("array_col[0]") + .show(2) +``` + +### User-Defined Functions (UDF) + +One of the most powerful things that you can do in Spark is define your own functions. Functions that operate on the data, record by record. + +Performance considerations: + +- UDFs in Scala or Java, you can use it within the Java Virtual Machine (JVM) +- In Python Spark starts a Python process on the workers and serializes all data to a format that Python understands. + +```scala +val udfExampleDF = spark.range(5).toDF("num") +def power3(number:Double):Double = number * number * number + +import org.apache.spark.sql.functions.udf +val power3udf = udf(power3(_:Double):Double) + +udfExampleDF.select(power3udf(col("num"))).show() + +// register + +spark.udf.register("power3", power3(_:Double):Double) +udfExampleDF.selectExpr("power3(num)").show(2) +``` + +## Aggregations + +groupings types in Spark: +(all return a RelationalGroupedDataset) + +- group by +- window +- grouping set +- rollup +- cube diff --git a/Apache/Apache Spark/_ jdbc.md b/Apache/Apache Spark/_ jdbc.md new file mode 100644 index 0000000..8601ce8 --- /dev/null +++ b/Apache/Apache Spark/_ jdbc.md @@ -0,0 +1,63 @@ +--- +title: '# jdbc' +updated: 2022-04-03 15:16:26Z +created: 2021-05-04 14:58:11Z +--- + +### method a load drivers + +```python +import os +os.environ['PYSPARK_SUBMIT_ARGS'] = '--jars file:/home/john/opt/jars/postgresql-42.2.5.jar pyspark-shell' +``` + +### method b load drivers + +```bash +pyspark \ +--packages org.postgresql:postgresql:42.2.5 \ +--driver-class-path /home/john/opt/jars/postgresql-42.2.5.jar +``` + +alone driver-class-path is also OK + +```python +from pyspark.sql import DataFrameReader, SparkSession + +spark = SparkSession.builder \ + .master("local") \ + .appName("jdbc data sources") \ + .config("spark.sql.shuffle.partitions", "4") \ + .getOrCreate() +``` + +### method 1 + +```python +df_company = ( + spark.read.format("jdbc") + .option("url", "jdbc:postgresql://172.17.0.2/postgres") + .option("dbtable", "public.company") + .option("user", "postgres") + .option("password", "qw12aap") + .option("driver", "org.postgresql.Driver") + .load() +) +df_company.show() +``` + +### method 2 + +```python +dataframe = ( + spark.read.format("jdbc") + .options( + url="jdbc:postgresql://172.17.0.2/postgres?user=postgres&password=qw12aap", + database="public", + dbtable="company", + driver="org.postgresql.Driver" + ) + .load() +) +dataframe.show() +``` diff --git a/Apache/Apache Spark/snippets.md b/Apache/Apache Spark/snippets.md new file mode 100644 index 0000000..d8bb029 --- /dev/null +++ b/Apache/Apache Spark/snippets.md @@ -0,0 +1,24 @@ +--- +title: snippets +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# snippets + +```python +df_main.join(df_sub,['finr','belastingjaar'],'left').filter(df_sub["element"].isin(20,30)).drop("element").groupBy('finr','belastingjaar').sum("waarde22").show() + +df_main.join(df_sub,['finr','belastingjaar'],'left').filter(df_sub["element"].isin(10,20)).show() +``` + +```python +df = spark.createDataFrame ([ +("a", 1, 10, "m1"), ("a", 1, 10, "m2"), ("a", 1, 30, "m3"), +("a", 1, 11, "m4")], +("a", "b", "cnt", "major")) +df.show() + +reshaped_df = df.groupby('a','b').pivot('major').max('cnt').fillna(0) +reshaped_df.show() +``` diff --git a/Apache/Apache Spark/spark_notes.md b/Apache/Apache Spark/spark_notes.md new file mode 100644 index 0000000..1c9d9d8 --- /dev/null +++ b/Apache/Apache Spark/spark_notes.md @@ -0,0 +1,495 @@ +--- +title: spark_notes +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Apache Spark Architecture + +Spark applications consists of + +- driver process + - maintaining information about the Spark Application; + - responding to a user’s program or input; + - analyzing, distributing, and scheduling work across the executors; +- executor processes + - executing code assigned to it by the driver; + - reporting the state of the computation on that executor back to the driver node; + +![Architecture Spark Application](../../_resources/architectureSparkApplication.png) + +## cluster manager + +controls physical machines and allocates +resources to Spark Applications: + +- Spark’s standalone cluster manager +- YARN +- Mesos + +Spark in _local mode_: driver and executor are simply processes. + +### Language APIs + +All languages (R, Python, Scala, Java) have similar performance characteristics when using structured API's. +When Python is using UDF then performance drops. Reason the Python code is executed in a separate Python process outside the JVM. + +Spark has two fundamental sets of APIs: + +- low-level "unstructured" (RDD) +- higher-level "structured" (Dataframe and Dataset) + +### Spark Session + +```scala +val myRange = spark.range(1000).toDF("number") +``` + +range of numbers represents a distributed collection: each part of this range of numbers exists on a different executor + +### Partitions + +To allow every executor to perform work in parallel, Spark breaks up the data into chunks called partitions. A partition is a collection of rows that sit on one physical machine in your cluster + +```scala +spark.conf.set("spark.sql.shuffle.partitions", "5") +``` + +Default is 200 partitions + +### Transformations + + Core data structures are immutable. To “change” , ie a DataFrame, you need to instruct Spark how you would like to modify it to do what you want. These instructions are called *transformations*. Return no output (**lazy evaluation**) This is because we specified only an abstract transformation, and Spark will not act on transformations until we call an action, but build up a plan of transformations (predicate pushdown) + +Types of transformations: + +- narrow: one input partition ==> one output partition +- wide: input partitions ==> many output partitions (shuffle= Spark writes to disk) ie aggregation and sort + +### Actions + +An action instructs Spark to compute a result from a series of transformations. ie count. + +Kind of actions: + +- view data in the console +- collect data to native objects in the respective language +- write to output data sources + +![Architecture Spark Application](../../_resources/ReadSortTakeDataframe-1.png) + +### logical plan + +The logical plan of transformations that we build up defines a lineage for the DataFrame so that at any given point in time, Spark knows how to recompute any partition by performing all of the operations it had before on the same input data + +### DataFrames and SQL + +Register any DataFrame as a table or +view (a temporary table) and query it using pure SQL. There is **no performance** difference between writing SQL queries or writing DataFrame code, they both “compile” to the same +underlying plan that we specify in DataFrame code + +```scala +flightData2015.createOrReplaceTempView("flight_data_2015") + +val sqlWay = spark + .sql("""SELECT DEST_COUNTRY_NAME, count(1) FROM flight_data_2015 GROUP BY DEST_COUNTRY_NAME +""") + +val dataFrameWay = flightData2015 + .groupBy('DEST_COUNTRY_NAME) + .count() +``` + +--- + +## Spark Toolset + +![Spark toolset](../../_resources/spark_toolset-1.png) + +### Running Production Applications (spark-submit) + +spark-submit does one thing: it lets you send your application code to a cluster and launch it to execute there + +On local machine: + +```bash +## specify location of external jars +LIB=...... +JARS=$(files=("$LIB"/*.jar); IFS=,; echo "${files[*]}") + +./bin/spark-submit \ +--class org.apache.spark.examples.SparkPi \ +--master local \ +--jars $JARS \ +./examples/jars/spark-examples_2.11-2.2.0.jar 10 +``` + +### Datasets: Type-Safe Structured APIs + +Datasets: statically type code in Java and Scala + +Dataframes: distibuted collection of objects of type Row +Datasets: collection of typed objects ie Scala Seq + +Reason to use Datasets: especially attractive for writing large applications, with which multiple software engineers must interact through well-defined interfaces. + +```scala +case class Flight(DEST_COUNTRY_NAME: String, +ORIGIN_COUNTRY_NAME: String, count: BigInt) + +val flightsDF = spark.read.parquet("/data/ +2010-summary.parquet/") + +val flights = flightsDF.as[Flight] +``` + +Advantage of Dataset use is that when call collect or take on a Dataset, it will collect objects of the proper type in your Dataset, not DataFrame Rows. This makes it easy to get type safety and securely perform manipulation in a distributed and a local manner without code changes + +### Structured Streaming + +Structured Streaming is a high-level API for stream processing that became production-ready in Spark 2.2. +It also makes it easy to conceptualize because you can write your batch job as a way to prototype it and then you can convert it to a streaming job + +### Machine Learning and Advanced Analytics + +Machine learning algorithms in MLlib require that data is represented as numerical values. All machine learning algorithms in Spark take as input a Vector type + +### Lower-Level APIs + +Virtually everything in Spark is built on +top of RDDs. One thing that you might use RDDs for is to parallelize raw data that you have stored in memory on the driver machine. +RDDs are available in Scala as well as Python. However, they’re not equivalent. + +### Spark’s Ecosystem and Packages + +[Spark Packages](https://spark-packages.org/) + +--- + +## Structured APIs + +The Structured APIs are the fundamental abstraction that you will use to write the majority of your data flows. +DataFrames and Datasets represent immutable, lazily evaluated plans that specify what operations to apply to data residing at a location to generate some output. +Support schema on write and schema on read. + +Spark uses an engine called **Catalys** and Spark is a program language in it own. The majority of our manipulations will operate strictly on Spark types. + +Within the Structured APIs, two more APIs: + +- untyped Dataframes; typed at runtime +- typed Datasets at compile time + +The “Row” type is Spark’s internal representation of its optimized in-memory format for computation. DataFrames are simply Datasets of Type Row. + +### Columns + +Columns represent a simple type like an integer or string a complex type like an array or map, or a null value. + +### Spark Types + +To work with the correct Scala types, use the following: + +```scala +import org.apache.spark.sql.types._ +val b = ByteType +``` + +Te get the Spark type: +the Scala type ie: + +- Short ==> ByteType +- Int ==> IntegerType +- etc + +### Overview of Structured API Execution + +single structured API query steps: + +1. Write DataFrame/Dataset/SQL Code +2. If valid code, Spark converts this to a Logical Plan. +3. Spark transforms this Logical Plan to a Physical Plan, checking for optimizations along the way +4. Spark then executes this Physical Plan (RDD manipulations) on the cluster. + +overview +![Catalyst Optimizer](../../_resources/CatalystOptimizer-1.png) + +Logical plan is first created and represents a set of abstract transformations that do not refer to executors or drivers. This plan is unresolved because although your code might be valid, the tables or columns that it refers to might or might not exist. Spark uses the catalog, a repository of all table and DataFrame information, to resolve columns and tables in the analyzer. The analyzer might reject the unresolved logical plan if the required table or column name does not exist in the catalog. Packages can extend the Catalyst to include their own rules for domain-specific optimizations. + +![LogicalPlan Spark](../../_resources/LogicalPlanSpark-1.png) + +The physical plan, often called a Spark plan, specifies how the logical plan will execute on the cluster by generating different physical execution strategies and comparing them through a cost model. Physical planning results in a series of RDDs and transformations + +![PhysicalPlan Spark](../../_resources/PhysicalPlanSpark-1.png) + +### Execution + +Upon selecting a physical plan, Spark runs all of this code over RDDs. further optimizations at runtime, generating native Java bytecode that can remove entire tasks or stages during execution. + +--- + +## Basic Structured Operations + +### Schemas + +A schema is a StructType made up of: + +- a number of fields +- StructFields, that have a name +- type (Spark types) +- Boolean flag: column can contain missing or null values +- optionally specify associated metadata with that column + +The metadata is a way of storing information about this column (Spark uses this in its machine learning library). + +```scala +import org.apache.spark.sql.types.{StructField,StructType, StringType,LongType} +import org.apache.spark.sql.types.Metadata + +val myManualSchema = StructType(Array( +StructField("DEST_COUNTRY_NAME", StringType, true), +StructField("ORIGIN_COUNTRY_NAME", StringType, true), +StructField("count", LongType, false, +Metadata.fromJson("{\"hello\":\"world\"}")) +)) + +val df = spark + .read + .format("json") + .schema(myManualSchema) + .load("/data/2015-summary.json") +``` + +### Columns and Expressions + +You cannot manipulate an individual column outside the context of a DataFrame; you must use Spark transformations +within a DataFrame to modify the contents of a column. + +Different ways to construct and refer to columns: + +```scala +import org.apache.spark.sql.functions.{col, column} +col("someColumnName") +column("someColumnName") +$"someColumnName" +'someColumnName + +df.col("count") // use +``` + +**Columns are not resolved until we compare the column names with those we are maintaining in the catalog. Column and table resolution happens in the analyzer +phase.** + +### Expressions + +Columns are expressions. An expression is a set of transformations on one or more values in a record in a DataFrame. (a function that takes as input one or more column names, resolves them, and then potentially +applies more expressions to create a single value for each record in the dataset) + + Each row in a DataFrame is a single record as an object of type Row + +### Creating Rows + +Only DataFrames have schemas. Rows themselves do not have +schemas. + +```scala +import org.apache.spark.sql.Row +val myRow = Row("Hello", null, 1, false) + +myRow(0) // type Any +myRow(0).asInstanceOf[String] // String +myRow.getString(0) // String +myRow.getInt(2) // Int +``` + +### Creating DataFrames + +```scala +val df = spark + .read + .format("json") + .load("/data/2015-summary.json") +df.createOrReplaceTempView("dfTable") +``` + +or + +```scala +import org.apache.spark.sql.Row +import org.apache.spark.sql.types.{StructField,StructType,StringType,LongType} + +val myManualSchema = new StructType(Array( +new StructField("some", StringType, true), +new StructField("col", StringType, true), +new StructField("names", LongType, false))) +val myRows = Seq(Row("Hello", null, 1L)) + +val myRDD = spark + .sparkContext + .parallelize(myRows) + +val myDf = spark + .createDataFrame(myRDD, myManualSchema) + +myDf.show() +``` + +### select and selectExpr + +```scala +mDF.select("colA","colB").show() +mDF.select('colA).show() +mDF.select(col("colA")).show() +mDF.select(expr("colA as aap")).show() // most flexible +mDF.select(expr("colA").alias("aap")).show() +mDF.selectExpr("colA as aap", "colB").show() // daily use; opens up the true power of Spark. +``` + +### Adding, renaming and dropping Columns + +Dataframe is NOT modified!! + +```scala +df.withColumn("numberOne", lit(1)) +df.withColumn("withinCountry", expr("ORIGIN_COUNTRY_NAME == DEST_COUNTRY_NAME")) + +df.withColumnRenamed("DEST_COUNTRY_NAME","dest") + +df.drop("DEST_COUNTRY_NAME") + +``` + +### Case Sensitivity + +By default Spark is case insensitive. Make sensitive: + +```scala +spark.sql("""set spark.sql.caseSensitive true""") +``` + +### Changing a Column’s Type (cast) + +```scala +df.withColumn("count2", col("count").cast("long")) +``` + +### Filtering Rows and Unique Rows + +```scala +df.filter(col("count") < 2).show(2) +df.where("count < 2").show(2) + +df.select("ORIGIN_COUNTRY_NAME", "DEST_COUNTRY_NAME") + .distinct().count() +``` + +### Random Samples and Split + +```scala +// Random Sample +val seed = 5 +val withReplacement = false +val fraction = 0.5 +df.sample(withReplacement, fraction, seed).count() + +// Random Splits +val dataFrames = df + .randomSplit(Array(0.25, 0.75), seed) +dataFrames(0) + .count() > dataFrames(1) + .count() // False +``` + +--- + +## Working with Different Types of Data + +**lit function**. This function converts a type in another language to its correspnding Spark representation. + +```scala +import org.apache.spark.sql.functions.lit +df.select(lit(5), lit("five"), lit(5.0)) +``` + +```scala +// where as in sql; === equal =!= not equal +vlucht.where(col("count") === 15) + .select("*") + .show(10) + +// best way +vlucht.where("count = 15").show(10) + +// column aap boolean; == equal +vlucht.selectExpr("*","count == 15 as aap").show(10) +``` + +**Better just write SQL!!!!** + +```scala +// compute summary statistics +df.describe().show() +``` + +### Working with Dates and Timestamps + +There are dates, which focus exclusively on calendar dates, and timestamps, which include both date and time information. Spark’s TimestampType class supports only second-level precision, which means that if you’re going to be working with milliseconds or microseconds, you’ll need to work around this problem by potentially operating on them as longs. + +### Working with Complex Types + +Structs: +Think of structs as DataFrames within DataFrames + +```scala +import org.apache.spark.sql.functions.struct +val complexDF = df.select(struct("Description","InvoiceNo") + .alias("complex")) + +complexDF.select(col("complex") + .getField("Description")) + .show() +``` + +split + +```scala +import org.apache.spark.sql.functions.split +df.select(split(col("Description"), " ") + .alias("array_col")) + .selectExpr("array_col[0]") + .show(2) +``` + +### User-Defined Functions (UDF) + +One of the most powerful things that you can do in Spark is define your own functions. Functions that operate on the data, record by record. + +Performance considerations: + +- UDFs in Scala or Java, you can use it within the Java Virtual Machine (JVM) +- In Python Spark starts a Python process on the workers and serializes all data to a format that Python understands. + +```scala +val udfExampleDF = spark.range(5).toDF("num") +def power3(number:Double):Double = number * number * number + +import org.apache.spark.sql.functions.udf +val power3udf = udf(power3(_:Double):Double) + +udfExampleDF.select(power3udf(col("num"))).show() + +// register + +spark.udf.register("power3", power3(_:Double):Double) +udfExampleDF.selectExpr("power3(num)").show(2) +``` + +## Aggregations + +groupings types in Spark: +(all return a RelationalGroupedDataset) + +- group by +- window +- grouping set +- rollup +- cube diff --git a/Apache/Camel.md b/Apache/Camel.md new file mode 100644 index 0000000..75a2e19 --- /dev/null +++ b/Apache/Camel.md @@ -0,0 +1,13 @@ +--- +title: Camel +updated: 2022-05-24 19:42:30Z +created: 2022-05-24 19:38:56Z +--- + +Apache Camel is an Open Source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data. + +Apache Camel™ is a versatile open-source integration framework based on known Enterprise Integration Patterns. + +Camel empowers you to define routing and mediation rules in a variety of domain-specific languages (DSL, such as Java, XML, Groovy, Kotlin, and YAML). This means you get smart completion of routing rules in your IDE, whether in a Java or XML editor. + +[source](https://camel.apache.org/) \ No newline at end of file diff --git a/Apache/Cassandra.md b/Apache/Cassandra.md new file mode 100644 index 0000000..3259899 --- /dev/null +++ b/Apache/Cassandra.md @@ -0,0 +1,12 @@ +--- +title: Cassandra +updated: 2022-05-24 19:28:24Z +created: 2022-05-24 19:26:51Z +--- + +What is Apache Cassandra? +> Apache Cassandra is an open source NoSQL distributed database trusted by thousands of companies for scalability and high availability without compromising performance. Linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure make it the perfect platform for mission-critical data. + +### Distributed +> Cassandra is suitable for applications that can’t afford to lose data, even when an entire data center goes down. There are no single points of failure. There are no network bottlenecks. Every node in the cluster is identical. + diff --git a/Apache/Flink.md b/Apache/Flink.md new file mode 100644 index 0000000..2617b6e --- /dev/null +++ b/Apache/Flink.md @@ -0,0 +1,45 @@ +--- +title: Flink +updated: 2022-05-24 19:01:38Z +created: 2022-05-24 18:44:47Z +--- + +# Stateful Computations over Data Streams + +![de4bf8596cb1518879ba2589540b3c7d.png](../_resources/de4bf8596cb1518879ba2589540b3c7d.png) + +Apache Flink is a framework and **distributed processing** engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments, perform computations at in-memory speed and at any scale. + +* * * +### Streaming use cases +- Event-driven Applications +- Stream & Batch Analytics +- Data Pipelines & ETL +* * * +### Guaranteed correctness +- Exactly-once state consistency +- Event-time processing +- Sophisticated late data handling +* * * +### Layered APIs +- SQL on Stream & Batch Data +- DataStream API & DataSet API +- ProcessFunction (Time & State) +* * * +### Excellent Performance +- Low latency +- High throughput +- In-Memory computing +* * * +### Scales to any use case +- Scale-out architecture +- Support for very large state +- Incremental check-pointing +* * * +### Data can be processed as unbounded or bounded streams. +- **Unbounded streams have a start but no defined end.** They do not terminate and provide data as it is generated. Unbounded streams must be continuously processed, i.e., events must be promptly handled after they have been ingested. It is not possible to wait for all input data to arrive because the input is unbounded and will not be complete at any point in time. Processing unbounded data often requires that events are ingested in a specific order, such as the order in which events occurred, to be able to reason about result completeness. +- **Bounded streams have a defined start and end**. Bounded streams can be processed by ingesting all data before performing any computations. Ordered ingestion is not required to process bounded streams because a bounded data set can always be sorted. Processing of bounded streams is also known as batch processing. + + + +[source](https://flink.apache.org/) \ No newline at end of file diff --git a/Apache/Fluentd.md b/Apache/Fluentd.md new file mode 100644 index 0000000..b3e4132 --- /dev/null +++ b/Apache/Fluentd.md @@ -0,0 +1,9 @@ +--- +title: Fluentd +updated: 2022-05-24 19:54:57Z +created: 2022-05-24 19:53:01Z +--- + +Fluentd is an open source data collector, which lets you unify the data collection and consumption for a better use and understanding of data. + +![c2a9b03791fddf1aabea180f18076f55.png](../_resources/c2a9b03791fddf1aabea180f18076f55.png) \ No newline at end of file diff --git a/Apache/Flume-1.md b/Apache/Flume-1.md new file mode 100644 index 0000000..b8e19f2 --- /dev/null +++ b/Apache/Flume-1.md @@ -0,0 +1,10 @@ +--- +title: Flume +updated: 2022-05-24 19:52:11Z +created: 2022-05-24 19:50:29Z +--- + +Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on **streaming data** flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application. + +![e3bb1a88acbda798341b1985f38d888c.png](../_resources/e3bb1a88acbda798341b1985f38d888c.png) + diff --git a/Apache/Flume.md b/Apache/Flume.md new file mode 100644 index 0000000..f66a1c6 --- /dev/null +++ b/Apache/Flume.md @@ -0,0 +1,51 @@ +--- +title: Flume +updated: 2022-05-24 18:29:31Z +created: 2021-05-04 14:58:11Z +--- + +# Apache Flume + +Streaming data into cluster +Developed with Hadoop in mind + +- Build-in sinks fir HDFS and Hbase +- Originally made to handle log aggregation + +***Flume is buffering data before delivering to the cluster.*** + +## Anatomy of a Flume Agent and Flow + +![Flume Agent](https://flume.apache.org/_images/DevGuide_image00.png?) + +### Three components of a Flume Agent: + +- Source + - Where data is comming from + - Optionally Channel Selectors and Interceptors + - Selectors: + - based on some selection the data is sent somewhere + - Interceptors: + - Data can add or reschape the data +- Channel + - how the data is transferred between Source and Sink (via memory or files) +- Sink + - Where the data is going + - multiple Sinks and can be organized into Sink Groups + - A Sink can connect to only ***one*** Channel + - Channel is notified te delete a message once the Sink processes it + +### Build-in Source Types: + +- Spooling directory, Avro (specific Hadoop format), Kafka, Exec (command-line), Thrift, Netcat (tcp/ip), HTTP, Custom, etc + +### Build-in Sink Types: + +- HDFS, Hive, HBase, Avro, Thrift, Elasticsearch, Kafka,Custom + +Flume Example +![example](../_resources/FlumeExample.png) + +First layer close to source and proces data. ie are in a local datacenter. +The second layer collects from and incests into the sink. +Between first amnd second layer of agents are source AVRO and Sink AVRO to transfer data very efficient. diff --git a/Apache/HBase.md b/Apache/HBase.md new file mode 100644 index 0000000..cd388d0 --- /dev/null +++ b/Apache/HBase.md @@ -0,0 +1,11 @@ +--- +title: HBase +updated: 2022-05-24 19:44:45Z +created: 2022-05-24 19:43:06Z +--- + +Apache HBase™ is the Hadoop database, a distributed, scalable, big data store. + +Use Apache HBase™ when you need random, realtime read/write access to your Big Data. This project's goal is the hosting of very large tables -- billions of rows X millions of columns -- atop clusters of commodity hardware. Apache HBase is an open-source, distributed, versioned, non-relational database modeled after Google's Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, Apache HBase provides Bigtable-like capabilities on top of Hadoop and HDFS. + +[Source](https://hbase.apache.org/) \ No newline at end of file diff --git a/Apache/Hadoop.md b/Apache/Hadoop.md new file mode 100644 index 0000000..aaecbd2 --- /dev/null +++ b/Apache/Hadoop.md @@ -0,0 +1,160 @@ +--- +title: Hadoop +updated: 2022-05-24 18:26:29Z +created: 2022-05-24 18:24:59Z +--- + +## **HDFS**: Hadoop Distributed File System + +### Hadoop 2 + +- high availability +- federations +- snapshots + +**YARN** was introduced in Hadoop version 2 to overcome scalability issues and resource management jobs. + +### Hadoop 3 + +- Overhead due to data replication factor + - Default replication factor of 3 + - fault-tolerant with better data locality and better load balancing of jobs among DataNodes + - overhead cost of around 200% + - less/not used data consumes resources + - Solution: **erasure coding**. This stores data durably while saving space significantly. +- YARN Timeline service re-architected +- YARN opportunistic containers & distributed scheduling, +- Optimizing map output collector: use of Java Native Interface (JNI) for optimazation. Useful for shuffle-intensive operations. +- higher availability factor of NameNode. While it is a highly available solution, in the case of the failure of an active (or standby) NameNode, it will go back to a non-HA mode. Support for ++more than one++ standby NameNode has been introduced. + +Dependency on Linux ephemeral port range: default ports are moved out of the ephemeral port range + +- Disk-level data skew: multiple disks (or drives) managed by DataNodes. Sometimes, adding or replacing disks leads to significant data skew within a DataNode. + +### Origins + +- Fault tolerant: The system should be able to handle any failure of the machines automatically, in an isolated manner. This means the failure of one machine should not affect the entire application. +- Load balancing: If one machine fails, then its work should be distributed automatically to the working machines in a fair manner. +- Data loss: once data is written to disk, it should never be lost even if one or two machines fail. + +### Concept of blocks and replication + +Blocks are created by splitting each file into 64 MB chunks (the size is configurable) and replicating each block three times by default so that, if a machine holding one block fails, then data can be served from another machine. + +### MapReduce model + +- Provide parallelism +- Fault tolerance +- Data locality features. +(Data locality means a program is executed where data is stored instead of bringing the data to the program.) + +**NameNodes** and **DataNodes** have a specific role in managing overall clusters. +NameNodes are responsible for maintaining metadata information. + +### Hadoop Logical View + +![Hadoop Logical View](../_resources/HadoopLogicalView.jpg) + +### Ingress/egress/processing + +- Ingesting (ingress) data +- Reading (Egress) data +- Processing already ingested data +These actions can be automated via the use of tools or automated code. + +### Data integration components + +For ingress/egress or data processing in Hadoop, you need data integration components. These components are tools, software, or custom code that help integrate the underlying Hadoop data with user views or actions. These components give end users a unified view of data in Hadoop across different distributed Hadoop folders, in different files and data formats. +ie Hue, Sqoop, Java Hadoop Clients, Hive, Beeline Clients + +### Data access interfaces + +Data access interfaces allow you to access underlying Hadoop data using different languages such as SQL, NoSQL, or APIs such as Rest and JAVA APIs, or using different data formats such as search data formats and streams. Sometimes, the interface that you use to access data from Hadoop is tightly coupled with underlying data processing engines. ie Spark SQL, SOLR or elastic search. + +### Data Processing Engines + +To manipulate underlying data and have different mechanisms to use system resources and have completely different SLA guarantees.ie MapReduce processing engine is more disk I/O-bound (keeping RAM memory usage under control) and it is suitable for batch-oriented data processing. Similarly, SPARK in a memory processing engine is less disk I/O-bound and more dependent on RAM memory. It is more suitable for stream or micro-batch processing. + +### Resource management frameworks + +Expose abstract APIs to interact with underlying resource managers for task and job scheduling in Hadoop. These frameworks ensure there is a set of steps to follow for submitting jobs in Hadoop using designated resource managers such as YARN or MESOS. These frameworks help establish optimal performance by utilizing underlying resources systematically. ie Tez or Slider. + +### Task and resource management + +sharing a large cluster of machines across different, simultaneously running applications in a cluster. YARN and MESOS. +YARN is a Unix process while MESOS is Linux-container-based + +### Data input/output + +The data input/output layer is primarily responsible for different file formats, compression techniques, and data serialization for Hadoop storage. + +### Data Storage Medium + +HDFS is the primary data storage medium used in Hadoop. It is a Java-based, high-performant distributed filesystem that is based on the underlying UNIX File System. + +## Core Hadoop Ecosystem + +![coreHadoop](../_resources/CoreHadoopEcosystem.png) + +Hadoop can handle big files effectively. Breaking files up in blocks of 64 or 128 mb (configurable) and stored across several commodity computers + + + + +## HDFS Read Mechanism + +![read](../_resources/HDFS_Read_Mechanism.png) + +## HDFS Write Mechanism + +![write](../_resources/HDFS_Write_mechanism.jpeg) + + +# Mapreduce + +## Conceptional + +1. Raw Data +2. Mapper +3. Shuffle and Sort (happens automatically by Hadoop) +4. Reducer + +![MapperReduce](../_resources/HadoopMapperReduce_conceptionally.png) + +## Distribution + +Raw data is splitup in partitions and partitions are distributed to different nodes + +## How all works together +![oveview](../_resources/OverviewMapReduceMasterNodes.png) + +Important is data locality. Client node stores data into HDFS. The DataNodes need to access this data, therefor the data has toe be distibuted efficient. + +## Handling Failure + +1. Application master monitors worker tasks for errors or hanging + +- restarts as needed +- Preferably on a diffent node + +1. application master goes down + +- YARN can try to restart it + +2. Entire node goes down + +- could be application master (1) +- resource manager will try to restart (Hadoop 3 has solution) + +1. Resource manager goes down + +- only when Zookeeper has a standby and will start an replace + + +# References + +[slides](https://www.slideshare.net/Simplilearn/hadoop-architecture-hdfs-architecture-hadoop-architecture-tutorial-hdfs-tutorial-simplilearn) + +[Youtube for slides(1:31 hour)](https://www.youtube.com/watch?v=CI0QkZYsLmw) + +[The Hadoop Ecosystem Table](https://hadoopecosystemtable.github.io/) \ No newline at end of file diff --git a/Apache/NiFi.md b/Apache/NiFi.md new file mode 100644 index 0000000..5782744 --- /dev/null +++ b/Apache/NiFi.md @@ -0,0 +1,69 @@ +--- +title: NiFi +updated: 2022-05-24 18:29:11Z +created: 2022-05-21 13:19:51Z +--- + +## What is Apache NiFi used for: +- reliable and secure transfer of data between systems +- delivery of data from sources to analytics platforms => top use case +- enrichment and preparation of data: + - conversion between formats => on thing at the time (json => csv) + - extraction/parsing + - route decisions => get value of json field and make decision on that value: send json to system A other wise to system B + +## What is Apache NiFi **NOT** used for? +- distribution computation +- complex event processing +- joins / complex rolling window operations + +## Hadoop ecosystem integration examples + + +### HDFS ingest +- MergeContent + - merges into appropriately sized files for HDFS + - based on size, number of messages, and time +- UpdateAttribute + - sets the HDFS directory and filename + - use expression language to dynamically bin by date +- PutHDFS + - write FlowFile content to HDFS + - support conflict resolution strategy and Kerboros authentication +![c45b3dcdac107122793b14d8bdd76a0f.png](../_resources/c45b3dcdac107122793b14d8bdd76a0f.png) + +### HDFS Retrieval +- ListHDFS + - perioddically perform listing on HDFS directory + - produces FlowFile per HDFS file + - flow only contains HDFS path & filename +- FetchHDFS + - retriece a file form HDFS + - use incoming FlowFiles to dynamically fetch +![a6ea2a07d58fac8a6739c7379c1b92f6.png](../_resources/a6ea2a07d58fac8a6739c7379c1b92f6.png) + +### HBase integration +- HBAse ingest - single cell =? table, row id, col family and col qualifier + - FlowFile content becomes the cell value +- HBase Ingest - Full row + - Row id can be a field in JSON or FlowFile attribute + +## Kafka integration +- PutKafka + - Provide Broker and topic name + - publishes FlowFile content as one or more messages + - Ability to send large delimited content, slit into messages bu NiFi +- GetKafka +- Provide ZK connection string and topic name +- produces a FlowFile for each message consumed + +## Stream Processing Integration +![1ce08014a43470c07e5314f1d69c6771.png](../_resources/1ce08014a43470c07e5314f1d69c6771.png) + +- Spark Streaming - NiFi Spark Receiver +- Storm - NiFi Spout +- Flink - NiFi Source & Sink +- Apex - NiFi Input Operations & Output Operations +- and many more integrations available + +[NiFi Videos](https://nifi.apache.org/videos.html) \ No newline at end of file diff --git a/Apache/RabbitMQ.md b/Apache/RabbitMQ.md new file mode 100644 index 0000000..469e3e5 --- /dev/null +++ b/Apache/RabbitMQ.md @@ -0,0 +1,13 @@ +--- +title: RabbitMQ +updated: 2022-05-24 19:49:57Z +created: 2022-05-24 19:45:41Z +--- + +RabbitMQ is the most widely deployed open source message broker. + +### Asynchronous Messaging +> Supports multiple messaging protocols, message queuing, delivery acknowledgement, flexible routing to queues, multiple exchange type. + + +[source](https://www.rabbitmq.com) diff --git a/Apache/Samoa.md b/Apache/Samoa.md new file mode 100644 index 0000000..5a93412 --- /dev/null +++ b/Apache/Samoa.md @@ -0,0 +1,13 @@ +--- +title: Samoa +updated: 2022-05-24 18:34:24Z +created: 2022-05-24 18:30:37Z +--- + +# Scalable Advanced Massive Online Analysis + +Apache SAMOA is a platform for mining on big data streams. It is a distributed streaming machine learning (ML) framework that contains a programming abstraction for distributed streaming ML algorithms. + +Apache SAMOA enables development of new ML algorithms without dealing with the complexity of underlying streaming processing engines (SPE, such as Apache Storm and Apache S4). Apache SAMOA also provides extensibility in integrating new SPEs into the framework. These features allow Apache SAMOA users to develop distributed streaming ML algorithms once and to execute the algorithms in multiple SPEs, i.e., code the algorithms once and execute them in multiple SPEs. + +[samoa-project](https://samoa-project.net/) \ No newline at end of file diff --git a/Apache/Storm.md b/Apache/Storm.md new file mode 100644 index 0000000..406a37f --- /dev/null +++ b/Apache/Storm.md @@ -0,0 +1,70 @@ +--- +title: Storm +updated: 2022-05-24 19:15:01Z +created: 2021-05-04 14:58:11Z +--- + +# Why use Apache Storm? + +Apache Storm is a free and open source distributed real-time computation system. Apache Storm makes it easy to reliably process **unbounded streams of data**, doing for real-time processing what Hadoop did for batch processing. Apache Storm is simple, can be used with any programming language. + +Apache Storm has many use cases: real-time analytics, online machine learning, continuous computation, distributed RPC, ETL, and more. Apache Storm is fast: a benchmark clocked it at over a million tuples processed per second per node. It is scalable, fault-tolerant, guarantees your data will be processed, and is easy to set up and operate. + +Apache Storm integrates with the queueing and database technologies you already use. An Apache Storm topology consumes streams of data and processes those streams in arbitrarily complex ways, re-partitioning the streams between each stage of the computation however needed. Read more in the tutorial. + +- Real-time continuous streaming data on clusters +- Runs on top of Yarn +- Works on individual events (**NOT** micro-batches like Spark) + - Storm is a better solution to Spark streaming +- Storm is perfect for sub-second latency (fast) + +## Storm Topology + + + +- Streams consists of ___tuples___ that flow through +- Spouts are ___sources___ of stream data (from Kafka, Twitter, etc) +- ___Bolts___ process stream data as it's recieved + - transform, aggregate, write to database / HDFS + - So no final state. Data stream continous goes on an on forever +- Storm topology is a graph of spouts ans bolts tat process the stream + - can get complex (In Spark you get the DAG for free) + +## Storm Architecture + + + +- Nimbus is a single point of failure + - Job tracker + - can restart quickly witout loosing any data + - HA is available as a Nimbus backup server +- Zookeeper (in it self is HA) +- Supervisors are doing the work + +## Developing Storm applications + +- usually in Java + - Bolts may be directed through scripts in other languages + - Selling point of Storm, but in practice in Java +- Storm Core + - lower-level API for Storm + - "At-least-once" semantics (possibility of duplicated data) +- Trident + - Highlevel API for Storm <=== prefer + - "Exactly once" semantics + - After submitted, Storm runs forever - until explicitly stopped + +## Storm vs Spark Streaming + +Storm +- tumbling window + - ie all events in the past 5 sec exactly; no overlap of events +- sliding window + - can overlap by design + Storm only Java + +Spark + - graph, ML, micro-batch streaming + Spark in Scala and Python + +Kafka and Storm => perfect combination diff --git a/Cloud/1 Cloud General.md b/Cloud/1 Cloud General.md new file mode 100644 index 0000000..a8474e3 --- /dev/null +++ b/Cloud/1 Cloud General.md @@ -0,0 +1,160 @@ +--- +title: 1 Cloud General +updated: 2021-09-06 19:07:35Z +created: 2021-05-04 14:58:11Z +--- + +# Traditional IT Services Deployment Models + +1. On Premises solutions (CPE – Customer Premises Equipment) +- All equipment is located in your building +- All equipment is owned by you +- There are clear lines of demarcation – everything in the building is your +- responsibility, the connections between offices are your network service +- provider’s responsibility +- Equipment is CapEx +- New equipment will typically take over a week to deploy +- Equipment requires technology refreshes +- You need to consider redundancy + +

+ +2. Colocation or ‘Colo’ services +- A colocation centre or “colo", is a data center location where the owner of the facility rents out space to external customers +- You own your own server, storage, and networking equipment within the colo facility (CapEx cost) +- The facility owner provides power, cooling, and physical security for their customer’s server, storage, and networking equipment + +# Server Virtualization +

+ +1. Type 1 Hypervisors run directly on the system hardware (Bare Metal) + ie VMware ESXi, Microsoft Hyper-V, Red Hat KVM +2. Type 2 Hypervisors run on top of a host operating system + ie VMware Workstation, Player and Fusion, Virtualbox, Parallels, QEMU + +# Definition Cloud Computing NIST [1]: +> Cloud computing is a model for enabling ubiquitous, convenient, on- +demand network access to a shared pool of configurable computing +resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal +management effort or service provider interaction + +# Essential Characteristics +1. **On-demand self-service.** A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider. + +3. **Broad network access.** Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, tablets, laptops, and workstations). +>Comments: Servers can be quickly provisioned and decommissioned based on current demand. Elasticity allows customers to achieve cost savings and is often a core justification for adoption + +4. **Resource pooling.** The provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or data center). Examples of resources include storage, processing, memory, and network bandwidth. + +5. **Rapid elasticity.** Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be appropriated in any quantity at any time. + +6. **Measured service.** Cloud systems automatically control and optimize resource use by leveraging a metering capability1 at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service. + + +# Service Models: +## On Premise (green customer managed, orange Provider managed) +![](../_resources/onpremise.png) + +## Colo (green customer managed, orange Provider managed) +![](../_resources/colo.png) + +1. ## Software as a Service (SaaS) + The capability provided to the consumer is to use the provider’s applications running on a cloud infrastructure. The applications are accessible from various client devices through either a thin client interface, such as a web browser (e.g., web-based email), or a program interface. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings. + ![SaaS](../_resources/SaaS.png) + + > examples: + - Microsoft Office 365 + - Salesforce.com + - Intuit + - Adobe Creative Cloud + - Gmail + +2. ## Platform as a Service (PaaS) + + The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages, libraries, services, and tools supported by the provider. 3 The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly configuration settings for the application-hosting environment. ![PaaS](../_resources/PaaS.png) + > examples: + - AWS Elastic Beanstalk + - Microsoft Azure (offers both IaaS and PaaS services) + - Google Apps + - Salesforce Force.com + - IBM Bluemix + +3. ## Infrastructure as a Service (IaaS) + +The capability provided to the consumer is to rovision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, and deployed applications; and possibly limited control of select networking components (e.g., host firewalls). +![IAS](../_resources/IaaS.png) + +>Cloud Providers will often offer three distinct flavours of IaaS compute: + +- Virtual machines on shared physical servers + - different customers share same underlying physical servers + - least expensive + - least number of options vCPU, RAM and storage + - provision quickly + - most common deployed option +- Virtual machines on dedicated physical servers + - customer is guaranteed that the underlying physical server is dedicated to them + - more options vCPU, RAM and storage + - may be require to sign a minimum length contract +- Dedicated bare-metal physical servers + - customer is given access to their own physical servers + - hypervisor is NOT installed and managed by cloud provider + - customer can install hypervior by themself or OS directly on HW + - most expensive option + - may be require to sign a minimum length contract + - AWS doesn't offer this option +- Customers can mix and match between the three types + +## Optional Servide Model (not defined by NIST) XaaS + +> Many cloud providers also offer other ‘as a Services’ +These are sometimes described as XaaS ‘Anything as a Service’ +Examples include: + +- DaaS Desktop As A Service +- DRaaS Disaster Recopry As A Service +- BaaS Backup As A Service +- Storage As A Device +- .... many more .. + +# Deployment Models + +1. ## Private cloud + + The cloud infrastructure is provisioned for exclusive use by a single organization comprising multiple consumers (e.g., business units). It may be owned, managed, and operated by the organization, a third party, or some combination of them, and it may exist on or off premises. + - different than On Prem? + - On-Demant Self-Service + - Rapid Elastic + - Broad Network Acces + - Resource Pooling + - Measured Service + - orders a new server typically through a web portal + - company will use automation software + - Private Cloud is most suitable for large companies where the long term ROI and efficiency gains can outweigh the initial effort and cost to set up the infrastructure and automated workflows + +2. ## Community cloud + + The cloud infrastructure is provisioned for exclusive use by a specific community of consumers from organizations that have shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be owned, managed, and operated by one or more of the organizations in the community, a third party, or some combination of them, and it may exist on or off premises. + - least common deployment model + - similar to a traditional extranet, but with full shared data center services instead of just network connectivity between On Prem offices + +3. ## Public cloud + + The cloud infrastructure is provisioned for open use by the general public. It may be owned, managed, and operated by a business, academic, or government organization, or some combination of them. It exists on the premises of the cloud provider. + - examples + - AWS + - Microsoft Azure + - IBM Bluemix + - Salesforce + - Most common deployment model + +4. ## Hybrid cloud + + The cloud infrastructure is a composition of two or more distinct cloud infrastructures (private, community, or public) that remain unique entities, but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load balancing between clouds). + - Companies with limited Private Cloud infrastructure may ‘cloud burst’ into Public Cloud for additional capacity when required + - A company could also have Private Cloud at their main site and use Public Cloud for their Disaster Recovery location + +sources: +1. [The NIST Definition of Cloud Computing](https://csrc.nist.gov/publications/detail/sp/800-145/final + +2. [A Practical Introduction to Cloud Computing](https://www.udemy.com/introduction-cloud-computing) \ No newline at end of file diff --git a/Cloud/2 Intro GC.md b/Cloud/2 Intro GC.md new file mode 100644 index 0000000..9141f92 --- /dev/null +++ b/Cloud/2 Intro GC.md @@ -0,0 +1,133 @@ +--- +title: 2 Intro GC +updated: 2021-09-06 19:07:42Z +created: 2021-09-06 07:30:13Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + + +06/09/2021 11:49 +# Google Cloud Platform (GCP) Infrastructure + +![5a61dcd70f5bb58e422ce7b3ef30f1b2.png](../_resources/5a61dcd70f5bb58e422ce7b3ef30f1b2.png) + + +![c53ad46a0de9b5989d9c61f607aad517.png](../_resources/c53ad46a0de9b5989d9c61f607aad517.png) +[https://cloud.google.com/video-intelligence] + + +![87bb8cb9d5ec12045bbbf38c08b8c7a0.png](../_resources/87bb8cb9d5ec12045bbbf38c08b8c7a0.png) +No will not save us. Increase of computer power has decreased dramaticly, because of fundamental fysic limitations. +One solution is to limit the power consumption of a chip, and you can do that by building Application-Specific Chips or ASICs +![9816005cb285fba7d59895ab9686676c.png](../_resources/9816005cb285fba7d59895ab9686676c.png) +The **T**ensor **P**rocessing **U**nit or TPU is an ASIC specifically optimized for ML. It has more memory and a faster processor for ML workloads than traditional CPUs or GPUs. +ML model training and future engineering is one of the most time-consuming parts of any machine learning project + +## Elastic Storage with Google Cloud Storage + +Storage and Ciomputer Power (VM) are sepearated and independent from each other. This makes cloud computing different from desktop computing. + +### Create Cloud Storage: +- through UI (browser) +- CLI: gsutil mb -p [PROJECT NAME] -c [STORAGE CLASS] -l [BUCKET LOCATION] gs://[BUCKET NAME]/ + +mb : make bucket +![65d46691cee632fc49a1054a983fba7c.png](../_resources/65d46691cee632fc49a1054a983fba7c.png) +All classes have multi-region, dual-region, and region location options. They differ based on the access speed and the cost. +For data analysis workloads, it's common to use a standard storage bucket within a region for staging your data. Why do I say within a region? That's because you need the data to be available to your data processing computing resources, and these will often be within a single region. Co-locating your resources this way maximizes the performance for data-intensive computations and could reduce network charges. +l : EUROPE-WEST4 +Bucket names have to be globally unique, so you can use that project ID as a unique name for your bucket. + +![35424d82fe826bba9b7f251153ef6278.png](../_resources/35424d82fe826bba9b7f251153ef6278.png) +### But what's a project and organisation? +A project is a base-level organizing entity for creating and using resources and services for managing billing, APIs, and permissions. +**Zones and regions** physically organize the GCP resources you use, whereas **projects** logically organize them. Projects can be created, managed, deleted, even recovered from accidental deletions. + +**Folders** are another logical grouping you can have for collections of projects. __Having an organization is required to use folders__. What's an organization? +The organization is a root node of the entire GCP hierarchy. While it's not required, an organization is quite useful because it allows you to set policies that apply throughout your enterprise to all the projects and all the folders that are created in your enterprise. +Cloud Identity and Access Management, also called **IM or IAM**, lets you fine-tune access control to all the GCP resources you use. + +Moving data around use gsutil. Locally and in the cloud +![ce13e5abfb93d0f151b38e19a04cf13b.png](../_resources/ce13e5abfb93d0f151b38e19a04cf13b.png) + +## Networking +Google's data centers around the world are interconnected by this **private** full duplex Google Jupiter network. +The petabit bisectional bandwidth and separation of compute and storage. No need to do everything on a single machine or even a single cluster of machines with their own dedicated storage => fast-enough network. Locality within the cluster is not important. +This is where Edge points of presence: Google's Network, interconnects with the public Internet at more than 90 internet exchanges and more than 100 points of presence worldwide. Google's Edge caching network places content close to end-users to minimize latency. + +## Security: On-premise vs Cloud-native + +![e46733bcebbc39b3852081970b1ad9cd.png](../_resources/e46733bcebbc39b3852081970b1ad9cd.png) + +Communications over the internet to our public cloud services are encrypted in transit +- In-transit encryption +- Multiple layers of security +- Backed by Google security eq protect agains DoS-attacks + +Stored data is automatically encrypted at rest and distributed for availability and reliability. +eg BigQuery: +- BigQuery table data encrypted with keys (and keys are also encrypted) and provides own defined encryption keys. +- Monitor and flag queries for anomalous behavior +- limit data access with autorized views + + +## Big data and ML products + +![5f9685eb06bc6b913e29821e3c135f78.png](../_resources/5f9685eb06bc6b913e29821e3c135f78.png) + +**GFS** Google File System to handle sharding and storing petabytes of data at scale. +**MapReduce**: manage large-scale data processing across large clusters of commodity servers. Automatically parallelized and executed on a large cluster of these commodity machines. Disadvantage: developers have to write code to manage all of the infrastructure of commodity servers. +(**Apache Hadoop**: now used in many industries for a huge variety of tasks that all share the common theme of volume, velocity and variety of structured, and unstructured data) +**Bigtable**: solved problem of recording and retrieving millions of streaming user actions with high throughput (inspiration for Hbase or MongoDB) +**Dremel** took a new approach to big data processing where Dremel breaks data into small chunks called shards, and compresses them into a columnar format across distributed storage. It then uses a query optimizer to farm out tasks between the many shards of data and the Google data centers full of commodity hardware to process a query in parallel and deliver the results. The big leap forward here was that the service, automanagers data imbalances, and communications between workers, and auto-scales to meet different query demands, and as you will soon see, Dremel became the query engine behind BigQuery. +**Colossus**: next-generation distributed data store. +**Spanner** as a planet scale relational database. +**Flume** and **Millwheel** for data pipelines. +**Pub/Sub** for messaging. +**TensorFlow** for machine learning. +**TPU** (Hardware). + +## [Google Cloud Public Datasets](https://services.google.com/fh/files/misc/public_datasets_one_pager.pdf) +Facilitate access to high-demand public datasets, hosted in n BigQuery and Google Cloud Storage. +[Datasets](https://cloud.google.com/solutions/datasets) + +## Choosing the right approach + +Computer Engine is one instance is +Infrastructure as an Service (***IAAS***) +Maximum flexibility managed by user. + +Google Kubernetes Engine (***GKE***) is a cluster of engines running containers Containarization is packing code, highly portable and uses resources efficiently. GKE is an orchestrator + +App Engine: Platform a an Service (***PAAS***) +Use for long living applications and can autoscale. + +Cloud Functions: Serverless environment (***FAAS**) Executes code in response to events. + +![c3f2d93dc2ae40260e94b36083fc20a9.png](../_resources/c3f2d93dc2ae40260e94b36083fc20a9.png) + +## What you can do with Google Cloud + +[Google Customers solutions](https://cloud.google.com/customers) +For Products and Solutions, filter on big data analytics and also on machine learning. Select a customer use case that interests you, then answer these three questions. +1. what were the barriers or challenges the customer faced? The challenges are important, you want to understand what they were. +2. how were these challenges solved with a cloud solution? What products did they use? +3. what was the business impact? + +Example Architucture +![da8ec8c3f3b5290b4f2ec2c5fb3a7122.png](../_resources/da8ec8c3f3b5290b4f2ec2c5fb3a7122.png) + +## Key roles in a data-driven organization + +***Data engineers*** to build the pipelines and get you clean data. +***Decision makers***, to decide how deep you want to invest in a data-driven opportunity while weighing the benefits for the organization. +***Analysts***, to explore the data for insights and potential relationships that could be useful as features in a machine learning model. +***Statisticians***, to help make your data-inspired decisions become true data-driven decisions, with their added rigor. +***Applied machine learning engineers***, who have real-world experience building production machine learning models from the latest and best information and research by the researchers. +***Data scientists***, who have the mastery over analysis, statistics, and machine learning. +***Analytics managers*** to lead the team. +***Social scientists and ethicists*** to ensure that the quantitative impact is there for your project and, it's the right thing to do. +A single person might have a combination of these roles, but this depends on the size of your organization. + diff --git a/Cloud/3 Recommendation Systems.md b/Cloud/3 Recommendation Systems.md new file mode 100644 index 0000000..a5e3f2d --- /dev/null +++ b/Cloud/3 Recommendation Systems.md @@ -0,0 +1,39 @@ +--- +title: 3 Recommendation Systems +updated: 2021-09-07 18:33:42Z +created: 2021-09-06 16:26:21Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + +# Cloud SQL and Cloud Dataproc + +- Cloud SQL: managed relational database +- Cloud Dataproc: managed environment on which you can run Apache Spark + +### Why moving from on-premisis to cloud +- utilizing and tuning on-premise clusters is dfficult +- but also moving dedicated storage to off-cluster storage + +A core aspect of a **recommendation system** is that you need to train and serve it at scale. + +### what is managed? +................ + +### Recommendation Systems +The core pieces are: +- data +- the model +- infrastructure +to train and serve recommendations to users. + +A core tenet of machine learning is to let the model learn for itself what +the relationship is between the data that you have, like user preferences (labeled data), and the data that you don't have. A history of good labeled data is important. + +Machine learning scales much better because it doesn't require hard-coded rules. It's all automated. Learning from data in an automated way, that's what machine learning is. + +Machine learning recommentation model is essentially asking "Who is this user like?" Secondly, is this subjectively a iyrm that people tend to rate highly? The predicted rating is a combination of both these factors. +All things considered, the rating of an item for a particular user will be the average of the ratings of users like this user but it's calibrated with the quality of the item itself. +Updating the data can be in **batch**, because ratings of the items doesn't change on eq a daily basis. On the otherhand there is a lot of data that has to be updated in a **fault tolerant way** that can scale to **large datasets** ==> Apache Hadoop. +When the user logs on, we want to show that user the recommendations that we precomputed specifically for them. So we need a transactional way (so that while the user is reading these predictions, we can update the predictions table as well) to store the predictions. eg 1 million user with each 5 predictions = 5 miljion rows. A MySQL database is sufficient. \ No newline at end of file diff --git a/Cloud/WK 1 Modernizing Data Lakes and Data Warehouses wi.md b/Cloud/WK 1 Modernizing Data Lakes and Data Warehouses wi.md new file mode 100644 index 0000000..ac144b8 --- /dev/null +++ b/Cloud/WK 1 Modernizing Data Lakes and Data Warehouses wi.md @@ -0,0 +1,126 @@ +--- +title: WK 1 Modernizing Data Lakes and Data Warehouses with GCP +updated: 2021-09-18 21:24:35Z +created: 2021-09-18 17:26:31Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + +# Explore the role of a data engineer + +![5908a4d876bce77f6cc35e42ca230be3.png](../_resources/5908a4d876bce77f6cc35e42ca230be3.png) + +![73822f556381b2d599822c18a4e09d83.png](../_resources/73822f556381b2d599822c18a4e09d83.png) + +The point of a data pipeline is to make data-driven decisions +A data lake brings together data from across the Enterprise into a single location. the Purpose of a data lake is to make data accessible for analytics. + +![0532ef04db4483bf2be14245b2c02b4b.png](../_resources/0532ef04db4483bf2be14245b2c02b4b.png) +Cloud storage is blob storage. So you might need to think about the granularity of what you store. + +Cloud storage bucket is a good option for staging all of your raw data in one place before building transformation pipelines into your data warehouse. + +![36ea2fef8a287ca026801e2d45632c20.png](../_resources/36ea2fef8a287ca026801e2d45632c20.png) +Because of Google's many data center locations and high network availability, +storing data in a GCS bucket is durable and performed. As a data engineer, you will often use a cloud storage bucket as part of your data lake to store many different types of raw data files, CSV, JSON, Avro, parquet, etc. You could then load or query them directly from BigQuery, which is a data warehouse. +Other Google Cloud platform products and services can easily query and integrate with your bucket once you've got it set up and loaded with data. + +Common challanges encountered by data engineers: +- Access to data +A typical problem: data is scatered around different locations( eq departments with there own systems: need to know how to combine the data) +![0745b249204a6a1ab9b5aaafaa18d53b.png](../_resources/0745b249204a6a1ab9b5aaafaa18d53b.png) +- Data accuracy and quality +cleaning, formatting, and getting the data ready for insightsrequires that you build +extract transform load or ETL pipelines. **ETL pipelines** are usually necessary to ensure data accuracy and quality. The cleaned and transformed data are typically stored not in a data lake, but in a **data warehouse**. A data warehouse is a consolidated place just like a data lake, it's a consolidated place. But this time, the data that we're storing is all easily joinable and queryable. Unlike a data lake where the data is in a row format in the data warehouse, the data is stored in a way that makes it very efficient to query +- Availiability of computational resources +The problem is that the compute that's needed by any specific ETL job is not constant over time. This means that when traffic is low, you're going to be wasting money because you have computers out there doing nothing, and when traffic is high, those computers are so busy that your jobs are taking way too long. +- Query performance +Once your data is in your data warehouse, you need to optimize the queries that your users are running to make the most efficient use of your compute resources. + + +## BigQuery +**BigQuery** is Google Cloud's petabyte scale **Serverless Data Warehouse.** +Datasets are collections of tables that can be divided along business lines or a given analytical domain. Each dataset is tied to a GCP project. +![67abe953280b3a8006e5a19d4e094040.png](../_resources/67abe953280b3a8006e5a19d4e094040.png) +Cloud Identity and Access Management, or Cloud IAM is used to grant permission to perform specific actions BigQuery. This replaces the SQL grant and revoke statements that are used to manage access permissions in traditional SQL databases. +![362653ef1df287f348947de381643f48.png](../_resources/362653ef1df287f348947de381643f48.png) +BigQuery allocates storage and query resources dynamically based on your usage patterns. Storage resources are allocated as you consume them, and deallocated as you remove data or you drop tables. Query resources are allocated according to the query type and complexity. Each query uses some number of what are called slots. **Slots** are units of computation that comprise a certain amount of CPU and RAM. + +## Data Lakes and Data Warehouses +![f5d936aaf30e3634df936c86cd99003e.png](../_resources/f5d936aaf30e3634df936c86cd99003e.png) + +Considerations when choosing a data warehouse: +- The data warehouses going to serve as a sink. Will the data warehouse be fed by a batch pipeline or by a streaming pipeline? +Need to be up-to-the-minute correct or is it enough to load data into it once a day or once a week? +- Will the data warehouse scale to meet my needs? +- How is the data organized? is it cataloged? Is it access controlled? +Be able to share access. data to all your stakeholders? Who will pay for the querying? +- Is the warehouse design for performance? Carefully consider concurrent query performance,and whether that performance comes out of the box,or whether you ended go around creating indexes and tuning the data warehouse +- What level of maintenance is required by your engineering team? + +![7b7de3102acdf026a8f7d05aba8e7217.png](../_resources/7b7de3102acdf026a8f7d05aba8e7217.png) +BigQuery provides mechanisms for automated data transfer and powers business applications using technologies like SQL that teams already know and use + +### Other option +That is to treat BigQuery as just a query engine and allow it to query the data in the data lake, data in place. For example, you can use BigQuery to directly query database data in Cloud SQL, that is managed relational databases like PostgreSQL, MySQL, and SQL Server. You can also use BigQuery to directly query files and Cloud Storage as long as those files are in formats like CSV or parquet. +![920bc1ebbbe823a40736ed122e695036.png](../_resources/920bc1ebbbe823a40736ed122e695036.png) +The real power comes when you can leave your data in place and still join it against +other data in the data warehouse. + +## Transactional Databases vs Data Warehouses + - Cloud SQL backend transactional Database systems that support your company's applications: optimized to be a database for transactions. + ![703c231dfc0b7336315963917d74b038.png](../_resources/703c231dfc0b7336315963917d74b038.png) + - Data Warehouses that support your analytic workloads. Optimized for reads + + + ![94a60b4329d9d14e300dd2f806af1521.png](../_resources/94a60b4329d9d14e300dd2f806af1521.png) + + **The data Lake is designed for durability and high availability.** + + + ## How to provide access to the data warehouse while keeping to data governance best practices? + +The three most common clients are +1. Machine Learning engineers + - how long does it take for a transaction to make it from raw data all the way into the data warehouse? to be available at prediction time + - how difficult it would be to add more columns or + - making your data sets easily discoverable, documented and available to them to experiment on quickly +more rows of data into certain datasets? +3. data or business analysts + - rely on good clean data so that they can query it for insights and build dashboards. + - need datasets that have clearly defined schema definitions, the ability to quickly review rows, and the performance to scale too many concurrent dashboard users. + ![8bc3cae2d26662dcf3d103e60e365a3f.png](../_resources/8bc3cae2d26662dcf3d103e60e365a3f.png) +5. other data engineers + - always be available when we need it? + +## Google Cloud stackdriver +- monitoring tool +- track resource use +- create audit logs +- who has run what +- trace usage sensitive datasets +- create alerts and send notifications + +## Manage data access and governance +Overall governance of how data is to be used and not used by your users. Privacy and security, +Clearly communicating a data governance model for who should access and who should not be able to access. + +How will our end users discover different data sets that we have for analysis? +1. One solution for data governance is Cloud **Data Catalog** .Data Catalog makes all the metadata about your data sets available for users to search. +![c7b0d8e8356eaef330da4a5ba9040df3.png](../_resources/c7b0d8e8356eaef330da4a5ba9040df3.png) +2. **Data Loss Prevention API**. This helps you better understand and manage sensitive data. It provides fast scalable classification and reduction for sensitive data elements, like credit card numbers, names. + +## Productionize the data process +End-to-end and scalable data processing system +Data engineering team is responsible for the health of the plumbing, that is the pipelines and ensuring that the data is available and up-to-date for analytic and ML workloads. + +Common questions: +![43d7aa7e1b2fb5703bc8874d6295cf1c.png](../_resources/43d7aa7e1b2fb5703bc8874d6295cf1c.png) + +One common workflow orchestration tool used by enterprises is Apache Airflow and Google Cloud has a fully managed version of **Apache Airflow called Cloud Composer**. +Cloud Composer helps your data engineering team orchestrate the pieces to the data engineering puzzle that we discussed to date, and even more that we haven't come across yet. The power of this tool comes from the fact that GCP big data products and services have API endpoints that you can call. A Cloud Composer job can then run every night or every hour and kickoff your entire pipeline from broad data to the data lake and into the data warehouse for you. + + + + \ No newline at end of file diff --git a/Cloud/WK 2 Data wharehouse.md b/Cloud/WK 2 Data wharehouse.md new file mode 100644 index 0000000..78a2179 --- /dev/null +++ b/Cloud/WK 2 Data wharehouse.md @@ -0,0 +1,34 @@ +--- +title: WK 2 Data wharehouse +updated: 2021-09-20 11:29:16Z +created: 2021-09-20 09:08:49Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + +![4c3b142e5e73a6ac715600d0331150c2.png](../_resources/4c3b142e5e73a6ac715600d0331150c2.png) + +![d00c7365a714367186d44f165573aa33.png](../_resources/d00c7365a714367186d44f165573aa33.png) + +# BigQuery + +BigQuery organizes data tables into units called datasets +![3393db98a463262eb4c2453a6140b984.png](../_resources/3393db98a463262eb4c2453a6140b984.png) +The project is what the billing is associated with. +To run a query, you need to be logged into the GCP console. You'll run a query in your own GCP project and the query charges are then build to your project. +In order to run a query in a project, you need Cloud IAM permissions to submit a job. +Access control is through Cloud IAM, and is that the data set level and applies to all tables in the dataset. BigQuery provides predefined roles for controlling access to resources. By defining authorized views and row-level permissions to give different users different roles for for the same data. + +BigQuery data sets can be regional or multi-regional. + +![3a3ffb85892084804f4ad8c23681d1b2.png](../_resources/3a3ffb85892084804f4ad8c23681d1b2.png) + +Logs and BigQuery are immutable and are available to be exported to Stackdriver. + +# Loading data into BigQuery +EL, ELT, ETL +![9c2b3ab61d6331389b844684b029f52f.png](../_resources/9c2b3ab61d6331389b844684b029f52f.png) +If your data is an Avro format, which is self-describing BigQuery can determine the schema directly, if the data is in JSON or CSV format BigQuery can auto detect the schema, but manual verification is recommended. + +**Backfilling data** means adding a missing past data to make a dataset complete with no gaps, and to keep all analytic processes working as expected. \ No newline at end of file diff --git a/Cloud/WK1 Big Data and Machine Learning Fundamentals.md b/Cloud/WK1 Big Data and Machine Learning Fundamentals.md new file mode 100644 index 0000000..3fcbd72 --- /dev/null +++ b/Cloud/WK1 Big Data and Machine Learning Fundamentals.md @@ -0,0 +1,145 @@ +--- +title: WK1 Big Data and Machine Learning Fundamentals +updated: 2021-09-11 16:36:12Z +created: 2021-09-07 18:38:10Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + +07/09/2021 20:38 +# Migrating workloads to the cloud +as an example an Apache Hadoop, Spark platform and MySQL database +An migration needs to ad value +![410d3a1816f7f58c8d812d7047ada137.png](../_resources/410d3a1816f7f58c8d812d7047ada137.png) + +Reference of choosing the right storage access platform +![57f223b1256deb84e40fbefc1bc5960a.png](../_resources/57f223b1256deb84e40fbefc1bc5960a.png) + +**[Dataproc](https://cloud.google.com/dataproc)** is a fully managed and highly scalable service for running Apache Spark, Apache Flink, Presto, and 30+ open source tools and frameworks. + + - **Cloud Storage** as a global file system Data is unstructured. + - **[Cloud SQL](https://cloud.google.com/sql)** as an RDBMS Data is structured and transactions. +Cloud SQL generally plateaus out at a few gigabytes _One database_ +Fully managed relational database service for MySQL, PostgreSQL, and SQL Server. Advantages: + - Familiar + - Flexible pricing + - Managed backups + - Connect from anywhere + - Automatic replication + - Fast connection from GCE (Google Compute Engine) & GAE (Google App Engine) + - Google security + - **Cloud Datastore** as a transactional No-SQL object-oriented database. Key-Value pair. + - **Cloud BigTable** for high-throughput No-SQL append-only data. No transactions. A typical use case for Bigtable is sensor data for connected devices eg. +- **Cloud BigQuery** as a SQL data warehouse to power all your analytics needs +- **Cloud Spanner** transactional database that is horizontally scalable so that you can deal with data larger than a few gigabytes, or if you need multiple databases across different continents. + +Selection of storage in a visual way +- ![765bc3651f87f986301649e1efafd128.png](../_resources/765bc3651f87f986301649e1efafd128.png) + +## Challenge: Utilizing and tuning on-premise clusters + +One of the most common challenges from managing on premise Hadoop clusters is making sure they're efficiently utilized and tooled properly for all the workloads that their users throw at them +The problem here lies in the static nature of the on premise cluster capacity + +GCP think of clusters as flexible resources +Turn down clusters automaticly with scheduled deletion: +- idle time (minimum 10 minutes) +- timstamp (eq maximum x days) +- duration (granularity 1 second) +- shutdown when a job is finished + +![b4a66850e6f618c5b36f61b9efd52678.png](../_resources/b4a66850e6f618c5b36f61b9efd52678.png) +And you can use auto scaling as long as when you shut down the clusters node, it doesn't remove any data. So, you cannot store the data on the cluster, but that's why we store our data on Cloud Storage, or BigTable, or BigQuery, we store our data off cluster. So, **autoscaling works as long as you don't store your data in HDFS** + +![28f3c610f7afc0d7715e6bfa470edcdd.png](../_resources/28f3c610f7afc0d7715e6bfa470edcdd.png) +In addition to auto scaling, another advantage of running Hadoop clusters and GCP is that you can incorporate **preemptible virtual machines** into your cluster architecture.Preemptible VMs are highly affordable, shortlived compute instances that are suitable for batch jobs and fault tolerant workloads. +### Why fault tolerant? +Because preemptible machines, they offer the same machine types and +options as regular compute instances, but they last only after 24 hours and +they can be taken away whenever somebody else comes along and +offers a new compute needs for them. +So, if your applications are fault tolerant and Hadoop applications are, +then preemptable instances can reduce your **compute engine costs** significantly. Preemptible VMs are up to 80% cheaper than regular instances. +The pricing is fixed, you get an 80% discount. +**But just like autoscaling, preemptible VMs work when your workload +can function without the data being stored on the cluster!!!!** + +# Big Query +- BigQuery is actually two services in one, a fast SQL Query Engine and fully managed data storage for loading and storing your datasets +![200ebd8d5ccb744efcc58af1fa67988d.png](../_resources/200ebd8d5ccb744efcc58af1fa67988d.png) + + +The storage service and the query service work +together to internally organize the data to make your queries run +efficiently on terabytes and petabytes. +![56dff0bf25437b4d77b8a2504ff17ae8.png](../_resources/56dff0bf25437b4d77b8a2504ff17ae8.png) + +The **storage service** automatically manages the data that you ingest into the platform. Data is contained within a project in what are called datasets, which would have zero to many tables or views. The tables are stored as highly compressed columns.Each column of that table highly compressed in Google's internal Colossus file system, which provides durability and global availability. +All the data stored here is only accessible to you and your project team as governed by your access policy. +The storage service can do both bulk data ingestion and streaming data ingestion via the API. For streaming the max row size for a streaming insert is one megabyte and the maximum throughput is 100,000 records per second per project. + +BigQuery manages the storage and the metadata for your dataset, automatically replicated, backed up and set up to auto scale for your query needs +The **query service** runs interactive or batch queries that are submitted through the console, the BigQuery web UI, the BQ command-line tool, or via the REST API. +There are BigQuery connectors to other services such as Cloud Dataproc and Cloud Dataflow, which simplify creating those complex workflows between +BigQuery and other GCP data processing services. +The query service can also run query jobs and data contained in other locations +You can run queries on tables that are a CSV file, for example, that are hosted +somewhere else in Cloud storage. Native BigQuery storage is the fastest. +- Serverless service meaning that's fully managed. So don't have to worry about how BigQuery stores data on disk or how it autoscales machines for large queries. +- BigQuery is designed to be an easy-to-use data warehouse. +- BigQuery's default pricing model is pay as you go. Pay for the number +of bytes of data that your query processes and any other permanent data that's stored inside of BigQuery. Automatic caching of query results: don't end up paying for the same query returning the same data twice. +- Data in BigQuery is encrypted at rest by default. +- Controlling access to your data can be as granular as specific columns, +say any column tag with PII (Personally Identifiable Information) or specific rows +- BigQuery works in tandem with Cloud IAM to set these roles and permissions at a project level, and then inherited down to the BigQuery level. +- BigQuery as both a data warehouse and an Advanced Query Engine is foundational for your AI and ML workloads.It's common for data analysts, engineers, and data scientists to use BigQuery to store, transform, and then feed those large datasets directly into your ML models. +- Write ML models directly in BigQuery using SQL. +![5f01121f49a2b85234a4dde5f3c243fa.png](../_resources/5f01121f49a2b85234a4dde5f3c243fa.png) +It stores all the incoming data from the left and allows you to do your analysis and your model-building. +![626a552aa3d1eb5331b7e1a0ed5161af.png](../_resources/626a552aa3d1eb5331b7e1a0ed5161af.png) + +# Cloud Dataprep +After your transformation recipe is complete, when you run a Cloud Dataprep job, it farms out the work to Cloud Dataflow which handles the actual processing of your data pipeline at scale. The main advantage to Cloud Dataprep is for teams want to use a UI for data exploration, and want to spend minimal time coding to build their pipelines. Lastly, with Dataprep you can schedule your pipeline to run at regular preset intervals. But, if you prefer to do all of your SQL and exploration work inside of BigQuery, you can also now use SQL to setup scheduled queries by using the @run_time parameter, or the query scheduler and the BigQuery UI. + +# Data security +So your insights are only shared with those people who should actually +have access to see your data. As you see here in this table, BigQuery inherits data security roles that you and your teams set up in Cloud IAM. +![92c4ab63a9cc743176d498e462af41af.png](../_resources/92c4ab63a9cc743176d498e462af41af.png) +Keep in mind that default access datasets can be overridden on a per dataset basis. Beyond Cloud IAM, you can also set up very granular controls over your columns and rows of data in BigQuery using the new data catalog service +and some of the advanced features in BigQuery, such as authorized views. +- Dataset users should have the minimum permission needed for their role. +- use separate projects or datasets for different environments (Dev, QA, PRD) +- Audit roles periodically +Data Access Policy for your organization, and it should specify how and when and why data should be shared, and with whom. + +# ML using SQL with BigQuery +![9106b716ae276078eea921e3e03078ee.png](../_resources/9106b716ae276078eea921e3e03078ee.png) +Look at the type of label or special column of data that you're predicting. +Generally, if it's a numeric datatype ==> **forecasting** +String value ==> **classification** +This row is either in this class or this other class, two classes or more ==> **multi-class classification** + +**ML benchmark** is the performance threshold that you're willing to accept from your model before you even allow it to be near your production data. It's critical that you set your benchmark before you train your model. So you can really be truly objective in your decision making to use the model or not. +![a435b4325f8a97c83018933cc44dff54.png](../_resources/a435b4325f8a97c83018933cc44dff54.png) + +The End -to_end BQML Process +![d80f9b4acc1e1833c3fcb177f3a50dd8.png](../_resources/d80f9b4acc1e1833c3fcb177f3a50dd8.png) + +1. Create model +![1166787a12cf4602040c27e3863abb43.png](../_resources/1166787a12cf4602040c27e3863abb43.png) + +Inspect the model Weights +![7ad6809dc9b770b2bb4967f5c8795b2f.png](../_resources/7ad6809dc9b770b2bb4967f5c8795b2f.png) + +Evaluate the model +![0dc381da545d4b7f9885e08555d4445a.png](../_resources/0dc381da545d4b7f9885e08555d4445a.png) + +Make batch predictions with ML.PREDICT +![db70a1d3769f8483876fb44e83fbc35c.png](../_resources/db70a1d3769f8483876fb44e83fbc35c.png) + +BQML Cheatsheet +![d2d5237d7c278a8fe8ffbe3175b6a7bc.png](../_resources/d2d5237d7c278a8fe8ffbe3175b6a7bc.png) + +![60b4e3ed0ebd1c3fb65b18d812f5f8fb.png](../_resources/60b4e3ed0ebd1c3fb65b18d812f5f8fb.png) \ No newline at end of file diff --git a/Cloud/WK2 Data Lakes.md b/Cloud/WK2 Data Lakes.md new file mode 100644 index 0000000..7e36f41 --- /dev/null +++ b/Cloud/WK2 Data Lakes.md @@ -0,0 +1,145 @@ +--- +title: WK2 Data Lakes +updated: 2021-09-19 15:15:03Z +created: 2021-09-19 10:18:20Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + +# Introduction to Data Lakes +![4bd6a4b0fe00fd4735c0efd1b01892a1.png](../_resources/4bd6a4b0fe00fd4735c0efd1b01892a1.png) + +**Data sources**: originating system or systems that are the source of all of your data +**Data sinks**: build those reliable ways of retrieving and storing that data + +The first line of defense in an enterprise data environment is your **data lake** variety of formats, volume, and velocity + +**Data pipelines**: doing the transformations and processing + +**Orchestration layer**: coordinate efforts between many of the different components at a regular or an event driven cadence. (*Apache airflow*) + +It's so important to first understand what you want to do first, and then finding which of the solutions best meets your needs. + +# Data Storage and ETL options on GCP +![26d5943bae46c8a9bc9a4210eb56ed45.png](../_resources/26d5943bae46c8a9bc9a4210eb56ed45.png) +- Cloud SQL and Cloud Spanner for **relational data** +- Cloud Firestore and Cloud Bigtable for **nosql data**. + +The path the data takes depends on: +- where is the data comming from +- Volume +- Where it has to go +- How much processing is needed to arrive in the sink + + +The method that you use to **load the data** into the cloud +depends on how much transformation is needed from that raw data +Cases: +- readily ingested (**EL** => Extract and Load eq avro format) Think also about federated search +- **ELT** => Extract Load Transform. Data is not in the right form to load into the sink. Volume is not big. eq use SQL to do the transformation: select from source and insert into the destination. +- **ETL** => Extract Transform Load. Transformation is essential or reduces the volume significant before importing into the cloud. + +# Building a Data Lake using Cloud Storage +Google Cloud Storage: +- strong persistant +- share globally +- encrypted +- controlled and private if needed +- moderate latency and high troughput +- relative inexpensive +- Object store: binary objects regartless of what the data is containt in the objects +- in some extent it has system compatibilities (copy out/in of objects as it where files) Cloud storage uses the bucket name and the object name to simulate a file system + +Use cases: +- archive data +- save state of application when shutdown instance + +![cbffc74d82c2acc542253f2a050fa649.png](../_resources/cbffc74d82c2acc542253f2a050fa649.png) +The two main entities in cloud storage are **buckets** and +**objects** +- buckets are containers which hold objects + - identified in a single globally unique name space (no one else can use that name. till deletion and name is released) + - associated with a particular region or multiple regions + - For a single region bucket the objects are replicated across zones within that one region (low-latency) + - multiple requesters could be retrieving the objects at the same time from different replicas (high throughput) +- objects exist inside of those buckets and not apart from them. + - When an object is stored, cloud storage replicates that object, it'll then monitor the replicas and if one of them is lost or corrupted it'll replace it automatically with a fresh copy. (high durability) + - stored with metadata. Used for access control, compression, encryption and lifecycle management of those objects and buckets. + +![d9fd10fbec213e8ae877a457a76bfe82.png](../_resources/d9fd10fbec213e8ae877a457a76bfe82.png) +1. the location of that bucket, location is set when a bucket is created and it can never be changed. +2. have the location to be a dual region bucket? + Select one region and the data will be replicated to multiple zones within this region +3. need to determine how often to access or change your data. +**[Storage classes](https://cloud.google.com/storage/docs/storage-classes)**: archival storage, backups or disaster recovery + +Cloud storage uses the bucket name and the object name to simulate a file system +![d74445c2bc5cf347c94f572c4787c354.png](../_resources/d74445c2bc5cf347c94f572c4787c354.png) +In example: +bucket name is declass +object name is de/modules/O2/script.sh +the forward slashes are just characters in the name + +A best practice is to avoid the use of sensitive information as part of bucket names, because bucket names are in a global namespace. + +![4e05321688bdc9bd53da7809ff1e4961.png](../_resources/4e05321688bdc9bd53da7809ff1e4961.png) + +# Securing Cloud Storage +![5b5bd4d599ad5e479af873c9f714ee7b.png](../_resources/5b5bd4d599ad5e479af873c9f714ee7b.png) +1. **IAM** is set at the bucket level. +- provides project roles and bucket roles: + - bucket reader + - bucket writer + - bucket owner. + +In the ability to create and delete buckets and to set IAM policy, +is a **project level role**. +The ability to create or change access control lists is an **IAM bucket role**. **Custom roles** are also available. + +2. Access control lists (**ACL**) +- applied at the bucket level or to individual objects. +So it provides more fine-grained access control. +Access lists are currently enabled by default + +All data in Google Cloud is **encrypted at rest and in transit** and there is no way to turn off the encryption. +![ec27c52f379456fd1ccdcd586b69c7de.png](../_resources/ec27c52f379456fd1ccdcd586b69c7de.png) + +Which data encryption option you use generally depend on your business, legal and regulatory requirements. + +Two levels of encryption: data is encrypted using a data encryption key, and then the data encryption key itself is then encrypted using a key encryption key or a **KEK**. These KEKs are automatically rotated on a schedule that use the current KEK stored in Cloud KMS, or the Key management Service + +![1798e9a44636860cd20db3b66ede4949.png](../_resources/1798e9a44636860cd20db3b66ede4949.png) +The fourth encryption option is client-side encryption. Client-side encryption simply means that you've encrypted the data before it's uploaded and then you have to decrypt the data yourself before it's used. Google Cloud storage still performs GMEK, CMEK, or CSEK encryption on the object. + +**Data locking** is different from encryption. Where encryption prevents somebody from understanding the data, locking prevents them from modifying the data. + + # Storing All Sorts of Data Types + Cloud Storage not for transactional data or for Analytics unstructured data. + ![0c0746feb4d1a4cb7802f32da7da5a43.png](../_resources/0c0746feb4d1a4cb7802f32da7da5a43.png) + + ![ca233fcb9a340843cc102e395c3b0a38.png](../_resources/ca233fcb9a340843cc102e395c3b0a38.png) +Online Transaction Processing or **OLTP** +Online Analytical Processing or **OLAP** + +![396364a8c39247681880d65a1f9b9a8b.png](../_resources/396364a8c39247681880d65a1f9b9a8b.png) + +# Storing Relational Data in the Cloud +**Cloud SQL**: +- managed service for third-party RDBMSs (MySQL, SQL server, PostgresSQL) +- cost effective +- default choice for those OLTP +- fully managed +![da9fb26de9169d4567f32995e6380343.png](../_resources/da9fb26de9169d4567f32995e6380343.png) + +**Cloud Spanner**: +- globally distributed database. Updates from applications running in different geographic regions. +- database is too big to fit in a single Cloud SQL instance + +**Cloud Bigtable** +- really high throughput inserts, like more than a million rows per second or all sure low latency on the order of milliseconds, consider + +**Difference between fully managed and serverless:** +By fully managed, we mean that the service runs on a hardware that you can control.Dataproc is fully mananged +A serverless product that's just like an API that you're calling. BigQuery and Cloud Storage is serverless +![1863886204ff15acbf0544007ff3c91a.png](../_resources/1863886204ff15acbf0544007ff3c91a.png) \ No newline at end of file diff --git a/Cloud/Wk2 Big Data and Machine Learning Fundamentals.md b/Cloud/Wk2 Big Data and Machine Learning Fundamentals.md new file mode 100644 index 0000000..d873702 --- /dev/null +++ b/Cloud/Wk2 Big Data and Machine Learning Fundamentals.md @@ -0,0 +1,157 @@ +--- +title: Wk2 Big Data and Machine Learning Fundamentals +updated: 2021-09-12 20:50:42Z +created: 2021-09-11 16:35:51Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + +![9948507e5860dd40b34a9f10e6b370c2.png](../_resources/9948507e5860dd40b34a9f10e6b370c2.png) + +# Message-oriented architectures with Pub/Sub + +## Distributed Messages +- Streaming data from various devices + - issues: bad data, delayed data, no data +- Distributing event notifications (ex: new user sign up) + - other services to subscribe to new messages that we're publishing out +- Scalable to handle volumes + - needs to handle an arbitrarily high amount of data so we don't lose any messages coming in. +- Reliable (no duplicates) + - We need all the messages and also a way to remove any duplicates if found + +**Pub/Sub is a distributed messaging** service that can receive messages from a variety of different streams, **upstream data** systems like gaming events, IoT devices, applications streams, and more. +Pub/Sub will scale to meet that demand. +- Ensures at-least-once delivery and passes them to subscribing applications +- No provisioning, auto-everything +- Open API's +- Global by default +- End-to-end encryption + +![51c2fc2521e7435aba198a5c23584999.png](../_resources/51c2fc2521e7435aba198a5c23584999.png) +Upstream data starts in from the left and comes into those devices from all around the globe. It is then **ingested** into Cloud Pub/Sub as a first point of contact with our system. Cloud Pub/Sub reads, stores, and then publishes out +any subscribers of this particular topic. +Cloud Dataflow as a subscriber to this pub subtopic in particular and It will ingest and transform those messages in an inelastic streaming pipeline. +If you're doing analytics one common data sink is Google BigQuery. + +## Architecture of Pub/Sub (like Kafka) +A central piece of Pub/Sub is the **topic**. +There can be zero, one, or many publishers. +Zero, one or many many subscribers relating to any given Pub/Sub topic. +Completely decoupled from each other. + +## Designing streaming pipelines with Apache Beam +Design of the actual pipeline in code the actual implementation in +serving of that pipeline at scale in production: +- is the code compatible with both batch and streaming data: **YES** +- Does the pipeline code SDK support the transformations I need to do? **Likely** +- Does it have the ability to handle late data coming into the pipeline? +- Are there any existing templates or solutions that we can leverage to quickly get us started? **Choose form templates** + +## What is Apache Beam? +Apache Beam is a **portable data processing programming model**. +- It's extensible (write and share new SDK's, IO connectors and transformation libraries) and Open Source +- can be ran in a **highly distributed** fashion +- It's **unified**: use a single model programming model for both batch and streaming use cases. +- **Portable**: Execute pipelines on multile excecution environments. No vendor lockin. +- You can browse and write your own connectors +- Build transformation libraries too if needed. +- Apache Beam pipelines are written in Java, Python or Go. +- The SDK provides a host of libraries for transformations and existing data connectors to sources and sinks. +![5151c1c2b55de604999b2b6f38822b08.png](../_resources/5151c1c2b55de604999b2b6f38822b08.png) +- Apache Beam creates a model representation of your code, which is portable across many runners. Runners pass off your model to an execution environment, which you could run in many different possible engines. +Cloud Dataflow is one of the popular choices for running Apache Beam as an engine. + +Example of an pipeline +![a6d9be269ce6bc83c9f5d824ef667013.png](../_resources/a6d9be269ce6bc83c9f5d824ef667013.png) +Transformations can be done in parallel, which is how you get that truly elastic pipeline +You can get input from many different and even multiple sources concurrently, +then you can write output to many different sinks, and the pipeline code remains the same. + +## Implementing streaming pipelines on Dataflow +![1238eb4c4d458a3679c68dff091d8e0c.png](../_resources/1238eb4c4d458a3679c68dff091d8e0c.png) + +![5ea764eaeeb247f7b43501c4d6e11653.png](../_resources/5ea764eaeeb247f7b43501c4d6e11653.png) + +Many Hadoop workloads can be done easily and more maintainably with Dataflow. Plus, Dataflow is serverless and designed to be NoOps. + +What do we mean by **serverless**? It means that Google will manage +all the infrastructure tasks for you, like resource provisioning and performance tuning, as well as ensuring that your pipeline is reliable. +![2fa4ce2967893077a569d372aed8a3ff.png](../_resources/2fa4ce2967893077a569d372aed8a3ff.png) + +[Source for Google DataFlow templates:](https://github.com/GoogleCloudPlatform/Dataflowtemplates) + +Recap: +![87df2df0f990da8682065eeb7f8381a9.png](../_resources/87df2df0f990da8682065eeb7f8381a9.png) +QPS: Queries Per Second + +# Visualizing insights with Data Studio +The first thing you need to do is tell Data Studio the Data Source +A Data Studio report can have any number of data sources. +The Data Source picker shows all the data sources that you have access to. + +other people who can view the report can potentially see +all the data in +that Data Source if you share that data source with them. +Warning: Anyone who can edit the rapport can also use +all the fields from any added data sources to create new charts with them. + +# Creating charts with Data Studio +- Dimension chips are green. +**Dimensions** are things like categories or buckets of information. Dimension values could be things like names, descriptions, or other characteristics of a category. +- **Metric** chips are blue. +Metrics measure dimension values. Metrics represent measurements or +aggregations such as a sum, x plus y, a count, how many of x, or even a ratio, x over y. A calculated field can also be a dimension + +Data Studio uses Google Drive for sharing and storing files. + +Share andcollaborate your dashboards with your team. Google login is required to edit a report.. No login for viewing. +Keep in mind that when you share a report, if you're connected to an underlying data source like a BigQuery data set, Data Studio does not automatically grant +permissions to viewers and that data source if the viewer doesn't already +have them and this is for data security reasons. +After you share your report, users can interact with filters and sort, and then you can collect feedback on the usage of your report through Data Studio's native integration with Google Analytics. + +# Machine Learning on Unstructured Datasets +Comparing approaches to ML +- **Use pre-built AI**: Dialogflow or Auto ML (10-100 images per label) + - provided as services + - Cloud Translation API + - Cloud Natural Language API + - Cloud-Speech-toText + - Cloud Video intelligence API (recognizing content in motion and action video's) + - Cloud Vision API (recognizing content in still images) + - Dialogflow Enterprise Edition( to build chatbots) +- **Add Custom Models**: only when you have a lot of data, +like 100,000 plus to millions of examples. +- **Create new Models**: TensorFlow, Cloud AI, Cloud TPU + +**Dialogflow** is a platform for building natural and rich conversational experiences. It achieves a conversational user experience by handling the natural language understanding for you. +It has built-in **entity recognition** which enables your agent to identify entities and label by types such as person, organization, location, events, products, and media. Also **sentiment analysis** to give an understanding of the overall sentiment expressed in a block of text. Even **content classification**, allows you to classify documents in over 700 predefined categories like common greetings and conversational styles. It has **multi-language support** so you can analyze text in multiple languages. +Dialogflow works by putting all of these ML capabilities together which you can then optimize for your own training data and use case. +Dialogflow then creates unique algorithms for each specific conversational agent, which continuously learns and is trained and retrained as more and more users engage with your agent. +![95d94272c1cec38e84f0dd6d9a5c9e48.png](../_resources/95d94272c1cec38e84f0dd6d9a5c9e48.png) +### Dialogflow benefits for users: +- Build faster: + - start training with only a few examples + - 40+ pre-built agents +- Engage more efficiently + - build-in naltural language understanding + - multiple options to connect with backend systems + - Training and analytics +- Maximize reach + - Build once, deploy everywhere + - 20+ language supported + - 14 single-click platform integrations and 7 SDKs + +## Customizing pre-built models with AutoML +![c95aa196527b6a81b5ad0e3e6eb4d20d.png](../_resources/c95aa196527b6a81b5ad0e3e6eb4d20d.png) + +So **precision** is the number of photos correctly classified as a particular label divided by the total number of photos classified with that label. +**Recall** is number of photos classified as a particular label divided +by the total number of photos with that label. +![e9bdd75ea38db93bd06929d6d2371de2.png](../_resources/e9bdd75ea38db93bd06929d6d2371de2.png) + +![317aa12b279ad4a9b0e289d1c654c4a3.png](../_resources/317aa12b279ad4a9b0e289d1c654c4a3.png) + +![f7e03cc71bfb39115e9ccb74f9306f2e.png](../_resources/f7e03cc71bfb39115e9ccb74f9306f2e.png) \ No newline at end of file diff --git a/Databases/Databases/postgresql.md b/Databases/Databases/postgresql.md new file mode 100644 index 0000000..959126c --- /dev/null +++ b/Databases/Databases/postgresql.md @@ -0,0 +1,178 @@ +--- +title: postgresql +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Postgresql + +![postgresql](https://d1q6f0aelx0por.cloudfront.net/product-logos/a28dcd12-094d-4248-bfcc-f6fb954c7ab8-postgres.png?) + + +psql DBNAME USERNAME +\d \\ list all relations +\d tablename \\ ddl + +```bash +sudo -u postgres CREATEUSER +sudo -u postgres CREATEDB +``` + +```psql +psql# ALTER USER WITH ENCRYPTED PASSWORD ''; +psql# GRANT ALL PRIVILEGES ON DATABASE to ; +``` + +## create csv file + +```sql +COPY (SELECT * FROM "public".empbase) TO '/tmp/empbase.csv' WITH CSV header; +``` + +### Jupyter: +> first install: +> ipython-sql and psycopg2 + +```jupyter +* %load_ext sql +* %sql postgresql://john:****@localhost/testdb +* %%sql + - select * from aap; +``` + +## From python program + +```python +import psycopg2 +query = "select * from aap" + +try: + conn = psycopg2.connect("postgres://john:qw12aap@localhost:5432/testdb") +except psycopg2.OperationalError as e: + print('Unable to connect!\n{0}').format(e) + sys.exit(1) +finally: + print("connected") + +cur = conn.cursor() +cur.execute(query) +for x in cur.fetchall(): + print(x) +cur.close() +conn.close() +``` + +[Source PostgreSQL](https://www.postgresql.org/files/documentation/pdf/11/postgresql-11-A4.pdf) + +## Connection from Apache Spark example: + +```scala +val driver = "org.postgresql.Driver" +Class.forName(driver) + +val df = spark.sqlContext + .read + .format("jdbc") + .options(Map("url"->"jdbc:postgresql://172.17.0.2:5432/postgres", "user"->"postgres","password"->"qw12aap","driver"->driver,"dbtable"->"company")) + .load() +``` + +## Install Docker + + + + +## create persistant storage location + +```bash +mkdir -p $HOME/docker/volumes/postgres +``` + +## launch docker container + +```bash +docker run --rm --name pg-docker -e POSTGRES_PASSWORD=docker -d -p 5432:5432 -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data postgres +``` + +## connect to running container + +```bash +docker exec -it pg-docker /bin/bash +``` + +## inside container + +```bash +psql -h localhost -U postgres -d postgres +``` + +[Postgresql tutorial](https://www.tutorialspoint.com/postgresql) + +# notes + +psql DBNAME USERNAME + +\d \\ list all relations +\d tablename \\ ddl + +sudo -u postgres CREATEUSER +sudo -u postgres CREATEDB + +psql# ALTER USER WITH ENCRYPTED PASSWORD ''; +psql# GRANT ALL PRIVILEGES ON DATABASE to ; + +## create csv file: + +```sql +COPY (SELECT * FROM "public".empbase) TO '/tmp/empbase.csv' WITH CSV header; +``` + +## In jupyter + +> first install: +> ipython-sql and psycopg2 + +Use: + +* %load_ext sql +* %sql postgresql://john:****@localhost/testdb +* %%sql + - select * from aap; + +## From python program + +```python +import psycopg2 +query = "select * from aap" + +try: + conn = psycopg2.connect("postgres://john:qw12aap@localhost:5432/testdb") +except psycopg2.OperationalError as e: + print('Unable to connect!\n{0}').format(e) + sys.exit(1) +finally: + print("connected") + +cur = conn.cursor() +cur.execute(query) +for x in cur.fetchall(): + print(x) +cur.close() +conn.close() +``` + +sources: +https://www.postgresql.org/files/documentation/pdf/11/postgresql-11-A4.pdf + +## Connection from Apache Spark example: + +```scala +val driver = "org.postgresql.Driver" +Class.forName(driver) + +val df = spark.sqlContext + .read + .format("jdbc") + .options(Map("url"->"jdbc:postgresql://172.17.0.2:5432/postgres", "user"->"postgres","password"->"qw12aap","driver"->driver,"dbtable"->"company")) + .load() +``` diff --git a/Databases/ElasticSearch/Analysis.md b/Databases/ElasticSearch/Analysis.md new file mode 100644 index 0000000..6415678 --- /dev/null +++ b/Databases/ElasticSearch/Analysis.md @@ -0,0 +1,110 @@ +--- +title: Analysis +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Analysis + +### Analyses is performed by a analyser +- tokenizer: breaks sentence in tokens, position of the tokens, optional for a specific language +- token filter: filter out stopwords +- character filter + +Reader -> tokenizer -> token filter -> token + +### Where use analyses? +- query +- mapping parameter +- index setting + +Analyser is used in the mapping part +Example + +### Analysers +1. Standard + - max_token_length (default 255) + - stopwords (defaults \_none_) + - stopwords_path (path to file containing stopwords) + - keep numeric values +2. simple + - lowercase + - remove special characters (ie dog's -> [dog, s]) + - remove numeric values +3. whitespace + - breakes text into terms whenever it encounters a whitespace character + - no lowercase transformation + - takes terms as they are + - keeps special characters +4. keyword + - no configuration + - takes all text as one keyword +5. stop + - stopword, stopword_path +6. pattern + - stopword, stopword_path, pattern, lowercase + - regular expression +7. custom + - tokenizer, char_filter, filter + + +### Example with standard analyzer +```json +PUT /test_analyzer +{ + "settings": { + "analysis": { + "analyzer": { + "my_analyzer": { + "type": "standard", + "max_token_length": 5, + "stopwords": "_english_" + } + } + } + }, + "mappings": { + "properties": { + "spreker_1": { + "type": "keyword", + "analyzer" : "my_analyzer" <== or an other analyzer; so per field + } + } + } +} +``` + +```json +GET /test_analyzer/_analyze +{ + "analyzer": "my_analyzer", + "field": "spreker_1", + "text": ["What is the this builders"] +} +``` + + +### without mapping; pattern analyzer +```json +PUT /test_analyzer +{ + "settings": { + "analysis": { + "tokenizer": { + "split_on_words": { + "type" : "pattern", + "pattern": "\\W|_|[a-c]", <-==== seperator whitespace or _ or chars a,b,c + "lowercase": true + } + }, + "analyzer": { + "rebuild_pattern": { + "tokenizer" : "split_on_words", + "filter": ["lowercase"] + + } + } + } + } +} +``` \ No newline at end of file diff --git a/Databases/ElasticSearch/Delete_DSL.md b/Databases/ElasticSearch/Delete_DSL.md new file mode 100644 index 0000000..dd2be65 --- /dev/null +++ b/Databases/ElasticSearch/Delete_DSL.md @@ -0,0 +1,35 @@ +--- +title: Delete_DSL +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Delete DSL + +### delete document + +```json +​```json +{ +DELETE //_doc/ +} +​``` +``` + + + + + + + + + + + +### delete by query + +```json +{ +POST //_delete_by_query +{ +``` \ No newline at end of file diff --git a/Databases/ElasticSearch/ES_Docker.md b/Databases/ElasticSearch/ES_Docker.md new file mode 100644 index 0000000..36fcb34 --- /dev/null +++ b/Databases/ElasticSearch/ES_Docker.md @@ -0,0 +1,22 @@ +--- +title: ES_Docker +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Setup ES with Docker + +## get ES Docker container and start +```bash + docker pull elasticsearch:7.6.2 # lastest is not supported by ES + docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.6.2 # start a singel node +``` + +## Test if running correctly +```bash + curl -I -XHEAD localhost:9200 +``` + +[Link to ES with Kibana](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/docker.html#_pulling_the_image) + + diff --git a/Databases/ElasticSearch/Index.md b/Databases/ElasticSearch/Index.md new file mode 100644 index 0000000..7a2b846 --- /dev/null +++ b/Databases/ElasticSearch/Index.md @@ -0,0 +1,202 @@ +--- +title: Index +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + + + +# Indexes + + +### Create index without mapping + +```json +{ + PUT +} +``` + +### Delete index + +```json +{ + DELETE +} +``` + +### Mapping is the process of defining how a document, and the fields it contains, are stored and indexed. + +[Mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html) + +```json +{ + GET //_mapping +} + +{ + GET //_mapping/field/ +} +``` + +### create a new index with mapping (example) +- Can not change the number of shards after index iz created +- re-index is possible (worstcase) +- replicas can be added later + + +### Types of fields +- object (may contain inner objects; json docs are hierarchocal in nature) +- nested + + +```json +PUT /items +{ + "settings": { + "index": { + "number_of_shards": , + "number_of_replicas": + } + }, + "mappings": { + "properties": { + "name": { + "type": "keyword" + }, + "production_date": { + "type": "date" + }, + "location": { + "type": "geo_point" + }, + "max_spots": { + "type": "integer" + }, + "description": { + "type": "text" + } + } + } +} +``` +[Field datatypes](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html) + + +## Elasticsearch Query DSL +```http + PUT /catalog + { + "settings": { + "index": { + "number_of_shards": , + "number_of_replicas": + } + }, + "mappings": { + "properties": { + "speaker": {"type": "keyword"}, + "play_name": {"type": "keyword"}, + "line_id": {"type": "integer"}, + "description": {"type": "text"} + "speech_number": {"type": "integer"} + } + } +} +``` + +### Adding a type mapping in an existing index +merged into the existing mappings of the _doc type +```json + PUT //_mapping + { + "properties": { + "name": { + "type": "text" + } + } + } +``` + + + + +### Create NESTED mapping, insert data and query +```json +DELETE /developer + +PUT /developer +{ + "mappings": { + "properties": { + "name": { + "type": "text" + }, + "skills": { + "type": "nested", + "properties": { + "language": { + "type": "keyword" + }, + "level": { + "type": "keyword" + } + } + } + } + } +} + +POST /developer/_doc/101 +{ + "name": "john Doe", + "skills": [ + { + "language": "ruby", + "level": "expert" + }, + { + "language": "javascript", + "level": "beginner" + } + ] +} + + +GET /developer/_search +{ + "query": { + "nested": { + "path": "skills", + "query": { + "bool": { + "must": [ + { + "match": { + "skills.language": "ruby" + } + }, + { + "match": { + "skills.level": "expert" + } + } + ] + } + } + } + } +} +``` + + + + + + +### Create OBJECT mapping, insert data and query + + + + + diff --git a/Databases/ElasticSearch/Python.md b/Databases/ElasticSearch/Python.md new file mode 100644 index 0000000..cccf7d1 --- /dev/null +++ b/Databases/ElasticSearch/Python.md @@ -0,0 +1,15 @@ +--- +title: Python +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# ElasticSearch with Python + + +Libraries: +- pyelasticsearch (DSL Queries) +- elasticutils (on top of the former) +- django-haystack + + diff --git a/Databases/ElasticSearch/Query_DSL.md b/Databases/ElasticSearch/Query_DSL.md new file mode 100644 index 0000000..0ae2988 --- /dev/null +++ b/Databases/ElasticSearch/Query_DSL.md @@ -0,0 +1,372 @@ +--- +title: Query_DSL +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Elasticsearch Query DSL + +### Queries can be classified into three types +1. Filtering by exact values +2. Searching on analyzed text +3. A combination of the two + +Every __document field__ can be classified: + +- either as an exact values +- analyzed text (also called full text) + +## Exact values +are fields like user_id, date, email_addresses +Querying documents can be done by specifying __filters over exact values__. Whether the document gets returned is a __binary__ yes or no +--- + +## Analyzed search +__Analyzed text__ is text data like product_description or email_body + +- Querying documents by searching analyzed text returns results based on __relevance__ (score) +- Highly complex operation and involves different __analyzer packages__ depending on the type of text data +- - The default analyzer package is the _standard analyzer_ which splits text by word boundaries, lowercases and removes punctuation +- less performant than just filtering by exact values + +## Expensive queries +1. Lineair scans + - script queries +2. high up-front + - fussie queries + - reqexp queries + - prefix queries without index_prefixes + - wildcard queries + - range queries on text and keyword fields +3. joinig queries + +4. Queries on deprecated geo shapes + +5. high per-document cost + + - script score queries + - percolate queries + +The execution of such queries can be prevented by setting the value of the `search.allow_expensive_queries` setting to `false` (defaults to `true`). + + +Queries behave different: **query context** or **filter context** + +| Queries | filters | +| --------------- | -------- | +| Fuzzy, scoring | Boolean | +| Slower | Faster | +| not Cachable | Cachable | + + +## Scoring queries +By default, Elasticsearch sorts matching search results by **relevance score**, which measures how well each document matches a query. But depends if the query is executed in **query** or **filter** context + + +## => Query context +“*How well does this document match this query clause?*” The relevance is stored in the **_score** meta_field +Query context is in effect whenever query clause is passed to the query parameter. + + + +## => Filter context +“*Does this document match this query clause?*” Answer is a true of false. No score is calculated == scoring of all documents is 0. + +Mostly used for filtering structured data, eq + +- Does this timestamp fall in range.... +- is the status field set to "text value" + +Frequently used filters will be cached + +Filter context in effect when filter clause is used + +- such as filter or must_not parameters in bool query +- filter parameter ins constant_score query +- filter aggregation + +Example +```json +GET /_search +{ + "query": { <= query context + "bool": { <= query context, together with matches: how well they match documents + "must": [ + { "match": { "title": "Search" }}, + { "match": { "content": "Elasticsearch" }} + ], + "filter": [ <= filter context + { "term": { "status": "published" }}, + { "range": { "publish_date": { "gte": "2015-01-01" }}} + ] + } + } +} +``` + +--- + +### Difference term vs match +- match : query aplies the same analyzer to the search at the time the data was stored +- term : does not apply any analyzer, so will look for exactly what is stored in the inverted index + +## The Query DSL + +Elasticsearch queries are comprised of one or many __Leaf query clauses__. Query clauses can be combined to create other query clauses, called __compound query clauses__. All query clauses have either one of these two formats: + +```json +{ + QUERY_CLAUSE: { // match, match_all, multi_match, term, terms, exists, missing, range, bool + ARGUMENT: VALUE, + ARGUMENT: VALUE,... + } +} + +{ + QUERY_CLAUSE: { + FIELD_NAME: { + ARGUMENT: VALUE, + ARGUMENT: VALUE,... + } + } +} +``` +Query clauses can be __repeatedly nested__ inside other query clauses + +```json +{ + QUERY_CLAUSE { + QUERY_CLAUSE: { + QUERY_CLAUSE: { + QUERY_CLAUSE: { + ARGUMENT: VALUE, + ARGUMENT: VALUE,... + } + } + } + } +} +``` + +## Two type of Query DSL (Leaf and Compound) + +### Leaf query clause +Look for a partiqulair value in a particulair field, such as match, term, range queries/ +These queries can be used by themselves. Use such as **match**, **term** or **range**. + +### Compound query clause +wrap other leaf(s) or compound queries and are used to combine multiple queries in a logical fashion (**bool** or **dis_max**) + +Or alter their behaviour (such as **constant_score**) + +- bool => must, must-not, should, filter, minimum_should_match + + multiple leaf or compound query clauses + + **must**, **should** => scores combined (), contributes to score + + **must_not**, **filter** => in context filter + + + **must** ==> like logical **AND**. + + **should** ==> like logical **OR**. + + You can use the `minimum_should_match` parameter to specify the number or percentage of `should` clauses returned documents *must* match. + + If the `bool` query includes at least one `should` clause and no `must` or `filter` clauses, the default value is `1`. Otherwise, the default value is `0` + + ```json + POST _search + { + "query": { + "bool" : { + "must" : { + "term" : { "user" : "kimchy" } + }, + "filter": { + "term" : { "tag" : "tech" } + }, + "must_not" : { + "range" : { + "age" : { "gte" : 10, "lte" : 20 } + } + }, + "should" : [ + { "term" : { "tag" : "wow" } }, + { "term" : { "tag" : "elasticsearch" } } + ], + "minimum_should_match" : 1, + "boost" : 1.0 + } + } + } + ``` + + + +- boosting query + +- constant_score query + +- dis_max query + +- function_score query + + + + +## Match Query Clause + +Match query clause is the most generic and commonly used query clause: +- run on a analyzed text field, it performs an analyzed search on the text +- run on an exact value field, it performs a filter +- calculates the score + +example: +```json +{ "match": { "description": "Fourier analysis signals processing" }} +{ "match": { "date": "2014-09-01" }} +{ "match": { "visible": true }} +``` + +## The Match All Query Clause + +Returns all documemts +```json +{ "match_all": {} } +``` + +## Term/Terms Query Clause +The term and terms query clauses are used to **filter** by a exact value fields by single or multiple values, respectively. In the case of multiple values, the logical connection is OR. + +```json +{ + "query": { + "term": { "tag": "math" } + } +} + +{ + "query": { + "term": { "tag": ["math", "second"] } + } +} + + + +``` + +## Multi Match Query Clause +Is run across multiple fields instead of just one + +```json +{ "query": { + "multi_match": { + "query": "probability theory", // value + "fields": ["title^3", "*body"], // fields, with wildcard * + // no fields == * + // title 3* more important + "type": "best_fields", + } + } +} +``` +[Other types](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html#multi-match-types) +## Exists and Missing Filters Query Clause +- The exists filter checks that documents have a value at a specified field +```json +{ + "query": { + "exists": { + "field": "*installCount" // also with wildcards + } +} +} +``` +- The missing filter checks that documents do not have have a value at a specified field + +```json +{ + "missing" : { + "field" : "title" + } +} +``` + +## Range Filter Query Clause +Number and date fields in ranges, using the operators gt gte lt lte +```json +{ "range" : { "age" : { "gt" : 30 } } } + +{ + "range": { + "born" : { + "gte": "01/01/2012", + "lte": "2013", + "format": "dd/MM/yyyy||yyyy" + } + } +} +``` + +## Query in filter context +### No scores are calculated: yes or no +The __query__ parameter indicates query context. +The __bool__ and two __match__ clauses are used in query context, which means that they are used to score how well each document matches. +The __filter__ parameter indicates __*filter context*__. Its term and range clauses are used in filter context. They will filter out documents which do not match, but they will __*not affect the score*__ for matching documents. +__Must__ clause is not required (score == 0.0) + +```json +GET /.kibana/_search +{ + "query": { + "bool": { + "must": [ + {"match": {"type" : "ui-metric"}}, + {"match": {"ui-metric.count" : "1"}} + ], + "filter": [ + {"range": {"updated_at": {"gte": "2020-04-01"}}} + ] + } + } +} +``` + +## Bool Query Clause +Are built from other query clauses are called compound query clauses. Note that compound query clauses can also be comprised of other compound query clauses, allowing for multi-layer nesting . + +The three supported boolean operators are __must__ (and) __must_not__ (not) and __should__ (or) +```json +{ + "bool": { + "must": { "term": { "tag": "math" }}, + "must_not": { "term": { "tag": "probability" }}, + "should": [ + { "term": { "favorite": true }}, + { "term": { "unread": true }} + ] + } +} +``` + +## Combining Analyzed Search With Filters +Example: query to find all posts by performing an analyzed search for “Probability Theory” but we only want posts with 20 or more upvotes and not those with that tag “frequentist”. +```json +{ + "filtered": { + "query": { "match": { "body": "Probability Theory" }}, + "filter": { + "bool": { + "must": { + "range": { "upvotes" : { "gt" : 20 } } + }, + "must_not": { "term": { "tag": "frequentist" } } + } + } + } +} +``` + +[Source: Understanding the Elasticsearch Query DSL](https://medium.com/@User3141592/understanding-the-elasticsearch-query-dsl-ce1d67f1aa5b) \ No newline at end of file diff --git a/Databases/ElasticSearch/cUrl_commands.md b/Databases/ElasticSearch/cUrl_commands.md new file mode 100644 index 0000000..5d9f610 --- /dev/null +++ b/Databases/ElasticSearch/cUrl_commands.md @@ -0,0 +1,111 @@ +--- +title: cUrl_commands +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# cUrl commands +```bash +curl -X '://:/?' -d '' +``` + +| Variables | Description | +|:------------- | :-------------| +| VERB | The appropriate HTTP method or verb. For example, GET, POST, PUT, HEAD, or DELETE | +| PROTOCOL | Either http or https. Use the latter if you have an HTTPS proxy in front of Elasticsearch or you use Elasticsearch security features to encrypt HTTP communications | +| HOST | The hostname of any node in your Elasticsearch cluster. Alternatively, use localhost for a node on your local machine | +| PORT | The port running the Elasticsearch HTTP service, which defaults to 9200 | +| PATH | The API endpoint, which can contain multiple components, such as _cluster/stats or _nodes/stats/jvm | +| QUERY_STRING | Any optional query-string parameters. For example, ?pretty will pretty-print the JSON response to make it easier to read | +| BODY | A JSON-encoded request body (if necessary) | + + +```bash +curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_bulk?pretty&refresh" --data-binary "@accounts.json" +``` + +```bash +curl "localhost:9200/_cat/indices?v" +``` + +## Test connection and ES correctly running +```bash + curl -I -XHEAD localhost:9200 +``` + +## Create Index +```bash +curl -X PUT http://localhost:9200/indexName +``` + +## Delete Index +``` bash +curl -X DELETE 'http://localhost:9200/indexName' +``` + +## List all indexes +``` bash +curl -X GET 'http://localhost:9200/_cat/indices?v' +``` + + +## query using URL parameters +### Lucene syntax +```bash +curl -X GET http://localhost:9200/IndexName/_search?q=school:Harvard +``` + +## Query using JSON +### ElasticSearch DSL syntax +```bash +curl -XGET --header 'Content-Type: application/json' http://localhost:9200/indexName/_search -d '{ + "query" : { + "match" : { "school": "Harvard" } + } +}' + +Lookup on index id +```bash +curl -XGET --header 'Content-Type: application/json' http://localhost:9200/indexName/_search -d '{ + "query" : { + "match" : { "_id": "37" } + } +}' +``` + +## List index mapping +### aka schema; fieldnames and their type +```bash +curl -X GET http://localhost:9200/indexName +``` + +## Add data +### indeName and doc# = 1 +curl -XPUT --header 'Content-Type: application/json' http://localhost:9200/indexName/_doc/1 -d '{ + "school" : "Harvard" +}' + + +## Update a document +## In this example create first a doc and then update the document +```bash +curl -XPUT --header 'Content-Type: application/json' http://localhost:9200/indexName/_doc/2 -d ' +{ + "school": "Clemson" +}' + +curl -XPOST --header 'Content-Type: application/json' http://localhost:9200/indexName/_doc/2/_update -d '{ +"doc" : { + "students": 50000} +}' +``` + +### load a dataset +```bash +curl -u elastic -H 'Content-Type: application/x-ndjson' -XPOST ':/bank/_bulk?pretty' --data-binary @accounts.json +``` + + + +[Source 1](https://www.bmc.com/blogs/elasticsearch-commands/) + diff --git a/Databases/ElasticSearch/examples.md b/Databases/ElasticSearch/examples.md new file mode 100644 index 0000000..94521ef --- /dev/null +++ b/Databases/ElasticSearch/examples.md @@ -0,0 +1,410 @@ +--- +title: examples +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# examples + + +```json +POST /items/_search +{ + "query": { + "term": {"age" : "10"} + } +} + +GET /items/_search +{ + "query": { + "match": { + "name": "jan" + } + } +} + +GET /shakespeare/_search + +GET /shakespeare/_search +{ + "query": { + "match": { + "speech_number": "1" + } + } +} + +GET /shakespeare/_search +{ + "query": { + "match_phrase": { + "text_entry": "scene I" + } + } +} + +GET /shakespeare/_search +{ + "query": { + "match_phrase_prefix": { + "text_entry": "with care" + } + } +} + +GET /shakespeare/_search +{ + "query": { + "match_all": {} + } +} + +GET /shakespeare/_search +{ + "query": { + "match_phrase": { + "text_entry": + {"query" : "shaken are", "slop": 2} + } +} +} + +GET /shakespeare/_search +{ + "query": { + "query_string": { + "fields": ["play_name", "speaker"], + "query": "KING HENRY IV", + "default_operator": "AND" + } + } +} + + + +GET /shakespeare/_search +{ + "query": { + "match": { + "line_id": 1 + } +} +} + +GET /shakespeare/_search +{ + "query": { + "terms": { + "speaker": [ + "KING HENRY IV", + "HENTY" + ] + } + } + } + + +GET /shakespeare/_search +{ + "query": { + "range": { + "line_id": { + "gte": 1, + "lte": 7 + } +} +} +} + + +GET /shakespeare/_search +{ + "query": { + "prefix": { + "speaker": { + "value": "KING" + } + } + } +} + + +GET /shakespeare/_search +{ + "query": { + "wildcard": { + "speaker": { + "value": "KING HENR*" + } + } + } +} + +GET /shakespeare/_mapping + + +GET /shakespeare/_search +{ + "query": { + "bool": { + "must": [ + {"term": { + "speaker": { + "value": "KING HENRY IV" + } + }} + ], + "filter": [ + {"term": { + "speech_number": "1" + }} + ] + } + } +} + + +GET /shakespeare/_search +{ + "query": { + "bool": { + "must": [ + {"match": { + "speaker": "KING HENRY IV" + }} + ], + "should": [ + {"term": { + "line_number": { + "value": "1.1.2" + } + }}, + {"term": { + "speech_number": { + "value": "2" + } + }} + ], + "minimum_should_match": 1, + "filter": [ + {"term": { + "play_name": "Henry IV" + }} + ] + } + + } + } + + +GET /shakespeare/_search +{ + "query": { + "bool": { + "should": [ + { + "wildcard": { + "line_number": { + "value": "1.1.?" + } + } + }, + { + "range": { + "line_id": { + "gte": 1, + "lte": 40 + } + } + } + ], + "minimum_should_match": 2 + } + } +} + +GET /shakespeare/_search +{ + "query": { + "query_string": { + "fields": ["speaker","play_name"], + "query": "KING HENRY IV", + "default_operator": "OR" + } + } +} + + + PUT /shakespeare/_mapping + { + "properties": { + "spreker_1": { + "type": "keyword" + } + } + } + + +DELETE /test_analyzer + + + +PUT /test_analyzer +{ + "settings": { + "analysis": { + "tokenizer": { + "split_on_words": { + "type" : "pattern", + "pattern": "\\W|_|[a-c]", + "lowercase": true + } + }, + "analyzer": { + "rebuild_pattern": { + "tokenizer" : "split_on_words", + "filter": ["lowercase"] + + } + } + } + } +} + + + "mappings": { + "properties": { + "spreker_1": { + "type": "keyword" + + } + } + } +} + +GET /test_analyzer/_analyze +{ + "analyzer": "rebuild_pattern", + "field": "spreker_1", + "text": ["Whsat is_the dd@this 1 builder's"] +} + +DELETE /developer + +PUT /developer +{ + "mappings": { + "properties": { + "name": { + "type": "text" + }, + "skills": { + "type": "nested", + "properties": { + "language": { + "type": "keyword" + }, + "level": { + "type": "keyword" + } + } + } + } + } +} + +POST /developer/_doc/101 +{ + "name": "john Doe", + "skills": [ + { + "language": "ruby", + "level": "expert" + }, + { + "language": "javascript", + "level": "beginner" + } + ] +} + + +GET /developer/_search +{ + "query": { + "nested": { + "path": "skills", + "query": { + "bool": { + "must": [ + { + "match": { + "skills.language": "ruby" + } + }, + { + "match": { + "skills.level": "expert" + } + } + ] + } + } + } + } +} + + + +PUT /developer1 +{ + "mappings": { + "properties": { + "name": { + "type": "text" + }, + "skills": { + "type": "object", + "properties": { + "language": { + "type": "keyword" + }, + "level": { + "type": "keyword" + } + } + } + } + } +} + +POST /developer1/_doc/101 +{ + "name": "john Doe", + "skills": [ + { + "language": "ruby", + "level": "expert" + }, + { + "language": "javascript", + "level": "beginner" + } + ] +} + + + +GET /developer1/_search +{ + "query": { + "match": { + "skills.language": "ruby" + } + } +} + + + +``` \ No newline at end of file diff --git a/Databases/ElasticSearch/http_commands.md b/Databases/ElasticSearch/http_commands.md new file mode 100644 index 0000000..2127285 --- /dev/null +++ b/Databases/ElasticSearch/http_commands.md @@ -0,0 +1,126 @@ +--- +title: http_commands +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# HTTP API commands + +### list indexes +```http + GET /_cat/indices?v + GET /_cat/indices + GET //_mapping + GET //_doc/ + + GET /_cat/health?v +``` +### Searches +```http + GET /,//_search + GET _search +``` + +### Most basic query; it returns all the content and with the score of 1.0 for every object. +```http +GET //_search +{ + "query":{ + "match_all":{} + } +} +``` + +### exanple +```http +GET /customer/_search +{ + "query": { + "match" : { "name" : "John Doe" } + } +} +``` + + +### Index API CRUD operations +```http + PUT /catalog/_doc/1 + PUT /// + POST /catalog/_doc{....} + POST // + + POST /catalog/_update/1{ doc { ....} } + POST ///_update + + DELETE // +``` + +### Creating an **new** index +## Elasticsearch Query DSL +```http + PUT /catalog + { + "settings": { + "index": { + "number_of_shards": , + "number_of_replicas": + } + }, + "mappings": { + "properties": { + "speaker": {"type": "keyword"}, + "play_name": {"type": "keyword"}, + "line_id": {"type": "integer"}, + "description": {"type": "text"} + "speech_number": {"type": "integer"} + } + } +} +``` + +### Adding a type mapping in an existing index +merged into the existing mappings of the _doc type +```http + PUT //_mapping + { + "properties": { + "name": { + "type": "text" + } + } + } +``` + +### Formatting the JSON response +```bash +curl -XGET http://localhost:9200/catalog/_doc/1?pretty=true +``` + +### Standard tokenizer +```http +POST _analyze + {"tokenizer": "standard", "text": "Tokenizer breaks characters into tokens!" +} +``` + +### analyzer example with english stopwords +```http +PUT index_standard_analyzer_english_stopwords +{ "settings": { + "analysis": { + "analyzer": { + "std": { + "type": "standard", + "stopwords": "_english_" } + } + } + }, + "mappings": { + "properties": { + "my_text": { "type": "text", "analyzer": "std" + } + } + } +} +``` + diff --git a/Databases/Neo4j/Applications with Python.md b/Databases/Neo4j/Applications with Python.md new file mode 100644 index 0000000..3d29502 --- /dev/null +++ b/Databases/Neo4j/Applications with Python.md @@ -0,0 +1,6 @@ +--- +title: Applications with Python +updated: 2022-08-08 19:51:06Z +created: 2022-08-08 19:50:49Z +--- + diff --git a/Databases/Neo4j/Complex Networks.md b/Databases/Neo4j/Complex Networks.md new file mode 100644 index 0000000..bddcedc --- /dev/null +++ b/Databases/Neo4j/Complex Networks.md @@ -0,0 +1,53 @@ +--- +title: Complex Networks +updated: 2021-10-04 17:28:56Z +created: 2021-10-04 15:37:12Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + +![193bfaaed851ce4eef6d43f0a20c03a7.png](../../_resources/193bfaaed851ce4eef6d43f0a20c03a7.png) +# Node and Edge Characteristics +![c9a1ce3aec2e82148c02a5d70d096109.png](../../_resources/c9a1ce3aec2e82148c02a5d70d096109.png) +![a546793b2366edc935ce4a1d98db7f1e.png](../../_resources/a546793b2366edc935ce4a1d98db7f1e.png) + +http://ks3329888.kimsufi.com/Intro2GraphTheoryAndComplexNetworksAnalysis/ + +![4d44a731b2dcebdb8e5210d5f6378b45.png](../../_resources/4d44a731b2dcebdb8e5210d5f6378b45.png) +1. Starting from a set of n vertices +2. Give two random vertices there is a probability P that they are linked together. + +![cf910f508994db7a3d68732db2c383fb.png](../../_resources/cf910f508994db7a3d68732db2c383fb.png) +1. Generate a lattice +2. Nodes are initially linked to k closest neighbours +3. Apply a rewiring probability + +![8cf3db7009d5518b37d9b0deb9b88dea.png](../../_resources/8cf3db7009d5518b37d9b0deb9b88dea.png) + +![5362730add08cb32ca4de62973425703.png](../../_resources/5362730add08cb32ca4de62973425703.png) + +![b685f0b119c20427c202a7143b227b60.png](../../_resources/b685f0b119c20427c202a7143b227b60.png) + +![f2a731b0ccf8dc8c852572318d9d6bea.png](../../_resources/f2a731b0ccf8dc8c852572318d9d6bea.png) + +![fe7d21e251098349149692c251f53dbc.png](../../_resources/fe7d21e251098349149692c251f53dbc.png) + +![536ce7d9e4c324339ba03a53d7eaaec9.png](../../_resources/536ce7d9e4c324339ba03a53d7eaaec9.png) + +![eb4e1a22065b0f7b00096573ca4f7cff.png](../../_resources/eb4e1a22065b0f7b00096573ca4f7cff.png) + +![418daa8979836e67e544c4b275820715.png](../../_resources/418daa8979836e67e544c4b275820715.png) + +![c30291ea3fd814cde29bbc5e2f275c3c.png](../../_resources/c30291ea3fd814cde29bbc5e2f275c3c.png) + +![0ceb8df673446ebaaa7f858751a50669.png](../../_resources/0ceb8df673446ebaaa7f858751a50669.png) + +[Network Analysis with Python and NetworkX Cheat Sheet](https://cheatography.com/murenei/cheat-sheets/network-analysis-with-python-and-networkx/) + +[Network Science Book](http://networksciencebook.com/) + +[The Colorado Index of Complex Networks](https://icon.colorado.edu/#!/) + +[Stanford Large Network Dataset Collection](https://snap.stanford.edu/data/) + diff --git a/Databases/Neo4j/Docker_neo4j.md b/Databases/Neo4j/Docker_neo4j.md new file mode 100644 index 0000000..c1c0f65 --- /dev/null +++ b/Databases/Neo4j/Docker_neo4j.md @@ -0,0 +1,29 @@ +--- +title: Docker_neo4j +updated: 2022-07-11 13:00:08Z +created: 2021-05-06 10:33:11Z +--- + +docker run \ + --name neo4jserver \ + -p7474:7474 \ + -p7687:7687 \ + -d \ + --rm \ + -v $HOME/work/data/neo4j/data:/data \ + -v $HOME/work/data/neo4j/logs:/logs \ + -v $HOME/work/data/neo4j/import:/var/lib/neo4j/import \ + -v $HOME/work/data/neo4j/plugins:/plugins \ + --user="$(id -u):$(id -g)" \ + --env NEO4J_AUTH=none \ + neo4j:latest + + + + + +``` +sudo groupadd -g 7474 neo4jgrp +getent group | grep neo4jgrp +sudo usermod -a -G neo4jgrp user_name +``` \ No newline at end of file diff --git a/Databases/Neo4j/Graph Data Modeling Fundamentals.md b/Databases/Neo4j/Graph Data Modeling Fundamentals.md index 52cbb00..eeabe5d 100644 --- a/Databases/Neo4j/Graph Data Modeling Fundamentals.md +++ b/Databases/Neo4j/Graph Data Modeling Fundamentals.md @@ -26,15 +26,15 @@ The Neo4j components that are used to define the graph data model are: Here are the steps to create a graph data model: -1. Understand the domain and define specific use cases (questions) for the application. -2. Develop the initial graph data model: +1. Understand the domain and define specific use cases (questions) for the application. +2. Develop the initial graph data model: a. Model the nodes (entities). b. Model the relationships between nodes. -3. Test the use cases against the initial data model. -4. Create the graph (instance model) with test data using Cypher. -5. Test the use cases, including performance against the graph. -6. Refactor (improve) the graph data model due to a change in the key use cases or for performance reasons. -7. Implement the refactoring on the graph and retest using Cypher. +3. Test the use cases against the initial data model. +4. Create the graph (instance model) with test data using Cypher. +5. Test the use cases, including performance against the graph. +6. Refactor (improve) the graph data model due to a change in the key use cases or for performance reasons. +7. Implement the refactoring on the graph and retest using Cypher. Graph data modeling is an iterative process. Your initial graph data model is a starting point, but as you learn more about the use cases or if the use cases change, the initial graph data model will need to change. In addition, you may find that especially when the graph scales, you will need to modify the graph (refactor) to achieve the best performance for your key use cases. @@ -67,7 +67,7 @@ When performing the graph data modeling process for an application, you will nee The data model describes the labels, relationships, and properties for the graph. It does not have specific data that will be created in the graph. Here is an example of a data model: -![0e5c55b7a519831b5ba0393544641782.png](../../images/0e5c55b7a519831b5ba0393544641782.png) +![0e5c55b7a519831b5ba0393544641782.png](../../_resources/0e5c55b7a519831b5ba0393544641782.png) There is nothing that uniquely identifies a node with a given label. A graph data model, however is important because it defines the names that will be used for labels, relationship types, and properties when the graph is created and used by the application. @@ -141,13 +141,11 @@ The main risk about fanout is that it can lead to very dense nodes, or supernode ## Properties for relationships -Properties for a relationship are used to enrich how two nodes are related. -When you define a property for a relationship, it is because your use cases ask a specific question about how two nodes are related, not just that they are related. +Properties for a relationship are used to enrich how two nodes are related. When you define a property for a relationship, it is because your use cases ask a specific question about how two nodes are related, not just that they are related. # Testing the Model You use the **use cases** to design the data model: - - includes labels for nodes - relationship types and direction - properties for the nodes and relationships. @@ -162,18 +160,15 @@ More data for testing is OK => test **scalability** The Cypher code used to test the use cases needs to be carefully reviewed for correctness. # Refactoring the Graph - ## Refactoring changing the data model and the graph. three reasons why refactor: - - The graph as modeled does not answer all of the use cases. - A new use case has come up that you must account for in your data model. - The Cypher for the use cases does not perform optimally, especially when the graph scales Steps (must) for refactoring: - 1. Design the new data model. 2. Write Cypher code to transform the existing graph to implement the new data model. 3. Retest all use cases, possibly with updated Cypher code. @@ -185,7 +180,6 @@ Limit the number of labels to 4 What is the primary reason to add labels to nodes is reduce the number of data accessed at runtime. ## Retesting After Refactoring - - After refactoring the graph, revisit all queries for all use cases. - Rewrite any Cypher queries for use cases that are affected by the refactoring. @@ -199,9 +193,8 @@ What is the primary reason to add labels to nodes is reduce the number of data a - avoid duplicating data in your graph - elilimnate duplication -> improve query performance - - In order to perform the query, all nodes must be retrieved to match a property. - - example refactoring list property to nodes - + - In order to perform the query, all nodes must be retrieved to match a property. + - example refactoring list property to nodes ``` MATCH (m:Movie) UNWIND m.languages AS language @@ -218,8 +211,7 @@ SET m.languages = null ## Eliminating Complex Data in Nodes Storing complex data in the nodes like this may not be beneficial for a couple of reasons: - -- Duplicate data. +- Duplicate data. - Queries related to the information in the nodes require that all nodes be retrieved. # Using Specific Relationships @@ -245,13 +237,13 @@ It has a **apoc.merge.relationship** procedure that allows you to **dynamically a relationship that connects more than two nodes. Mathematics allows this, with the concept of a hyperedge. Impossible in Neo4j. -![69b4c46435ed52c1fe5be0ba6a074be5.png](../../images/69b4c46435ed52c1fe5be0ba6a074be5.png) +![69b4c46435ed52c1fe5be0ba6a074be5.png](../../_resources/69b4c46435ed52c1fe5be0ba6a074be5.png) Email is new intermediate node -![ae805ac0f184fdb6cf93d6b038af28a9.png](../../images/ae805ac0f184fdb6cf93d6b038af28a9.png) +![ae805ac0f184fdb6cf93d6b038af28a9.png](../../_resources/ae805ac0f184fdb6cf93d6b038af28a9.png) -- Intermediate nodes deduplicate information. +- Intermediate nodes deduplicate information. - Connect more than two nodes in a single context. - Share data in the graph. - Relate something to a relationship. diff --git a/Databases/Neo4j/Graph Theory.md b/Databases/Neo4j/Graph Theory.md new file mode 100644 index 0000000..95a7b19 --- /dev/null +++ b/Databases/Neo4j/Graph Theory.md @@ -0,0 +1,76 @@ +--- +title: Graph Theory +updated: 2022-07-18 09:46:19Z +created: 2021-10-02 17:39:39Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + +## Euler Path +uses every single egde once; no repeats +No need to have start en end vertex the same. +**0 or 2 odd degree vertices, the rest even** +![0dbbbe727224fc349edc14517e3b93c9.png](../../_resources/0dbbbe727224fc349edc14517e3b93c9.png) + +## Euler Circuit +Every edge once; no repeats +Start and end vertex are the same +**All vertices must have an even degree** +Multiple Euler Circuits are possible and depends on the starting direction. +![3f0188b20328cc489d50b321df8d6bc7.png](../../_resources/3f0188b20328cc489d50b321df8d6bc7.png) + +## Fleury's Algorithme +Find an Euler Circuit, starting at vertex A +![2e79a268b8d9ef0e61db6ea008bafd7d.png](../../_resources/2e79a268b8d9ef0e61db6ea008bafd7d.png) +Remove the edge take to go the next vertex, but a disconnected graph is not allowed. Deleting an edge prevents backtracking. +![a99bbee6299b41776e45d4af26288c05.png](../../_resources/a99bbee6299b41776e45d4af26288c05.png) + +## Eulerization +Duplicate egdes so a Euler cirquit is in the graph, from and to vertices with odd degree. No new edges. Minimize duplication of edges. Multiple solutions possible. +![8cc8d717ee428ffeb48a396d804b91de.png](../../_resources/8cc8d717ee428ffeb48a396d804b91de.png) + +## Hamiltonian Circuit +Visit every vertex once, with no repeats, +![91c43645ccabeee0047757424141fb5d.png](../../_resources/91c43645ccabeee0047757424141fb5d.png) +Minimun cost Hamiltonian Circuit == (Traveling Salesman Problem) +Possible method: Brute Force +Optimal, bit not efficient +![5815ff14a02f76a5118a6ef8f86ed048.png](../../_resources/5815ff14a02f76a5118a6ef8f86ed048.png) + +## Complete Graph +All vertices are connected with all the others. +If n vertices then (n-1)!/2 different hamiltonian circuits + +Better then Brute Force the get the shortest circuit are heuristic methods; +- **Nearest Neighbor Algorithm** (not optimal) +Start at A and take at every vertex the edge with the lowest weight. (Greedy, doesn't look ahead) +![beb7e37bd0e389910f37d2ef559a1900.png](../../_resources/beb7e37bd0e389910f37d2ef559a1900.png) +- **Repeated Nearest Algorithm** +Same as Nearest Neighbor Algorithm but repeat for every vertex and select the lowest cost. +![89ef06b3ace7055254b5a4d1b2db1663.png](../../_resources/89ef06b3ace7055254b5a4d1b2db1663.png) +- **Sorted Edge Algorithm** (not optimal) +Add cheapest up, unless: + - a mini circuit (a curcuit that doesn't include all vertices) + - no vertex with a degree 3 + +AD = 1 +AC = 2 +AB = 4 (not because of degree 3 rule) +CD = 8 (not because creates mini circuit) +BD = 9 +BC = 13 +Most optimal is ADBCA = 25 +![b3fabbeba3f22c574c31078c58c5dbe8.png](../../_resources/b3fabbeba3f22c574c31078c58c5dbe8.png) + +## Hamiltonian Path +Visit every vertex once, with no repeats, +![c34ed20473a52dda3d546240f3b6e52c.png](../../_resources/c34ed20473a52dda3d546240f3b6e52c.png) + +## Kruskal's Algoritme (optimal and efficient) +Similar to Sorted Edge Algorithm, but no circuit. +Minimim cost spanning tree == every vertex is connected to an other vertex +![b55dc1138ff9d80431c97102da97c98c.png](../../_resources/b55dc1138ff9d80431c97102da97c98c.png) +Add cheapest up, unless it creates a circuit. +![c1e5b885c1aead4ff5700b0a10cc0302.png](../../_resources/c1e5b885c1aead4ff5700b0a10cc0302.png) + diff --git a/Databases/Neo4j/Intermediate Cypher Queries.md b/Databases/Neo4j/Intermediate Cypher Queries.md new file mode 100644 index 0000000..7210351 --- /dev/null +++ b/Databases/Neo4j/Intermediate Cypher Queries.md @@ -0,0 +1,590 @@ +--- +title: Intermediate Cypher Queries +updated: 2022-08-08 19:42:32Z +created: 2022-08-01 13:15:27Z +--- + +# Filtering Queries + +``` +CALL db.schema.visualization() +CALL db.schema.nodeTypeProperties() +CALL db.schema.relTypeProperties() +SHOW CONSTRAINTS +:HISTORY +:USE database +``` + +check multiple labels + +``` +match (p) +where p:Actor:Director +and p.born.year >= 1950 and p.born.year <= 1959 +return count(p) +``` + +``` +MATCH (p:Director)-[:DIRECTED]->(m:Movie)<-[:ACTED_IN]-(p) +WHERE "German" IN m.languages +return p.name, labels(p), m.title +``` + +``` +match (n)-[a]->(m:Movie) +where (n:Actor or n:Director) +and toUpper(a.role) contains 'DOG' +return n.name, m.title, a.role +``` + +### Difference EXPLAIN vs PROFILE + +- EXPLAIN provides estimates of the query steps +- PROFILE provides the exact steps and number of rows retrieved for the query. + +Providing you are simply querying the graph and not updating anything, it is fine to execute the query multiple times using **PROFILE**. In fact, as part of query tuning, you should _execute the query at least twice_ as the first execution involves the generation of the execution plan which is then cached. That is, the first PROFILE of a query will always be more expensive than subsequent queries. + +Useful use of exists to exclude patterns in the graph + +``` +MATCH (p:Person)-[:ACTED_IN]->(m:Movie) +WHERE p.name = 'Tom Hanks' +AND NOT exists {(p)-[:DIRECTED]->(m)} +RETURN m.title +``` + +If you profile this query, you will find that it is not performant, but it is the only way to perform this query. + +### Multiple MATCH Clauses + +``` +MATCH (a:Person)-[:ACTED_IN]->(m:Movie), + (m)<-[:DIRECTED]-(d:Person) +WHERE m.year > 2000 +RETURN a.name, m.title, d.name +``` + +In general, using a single MATCH clause will perform better than multiple MATCH clauses. This is because relationship uniquness is enforced so there are fewer relationships traversed. + +Same as above + +``` +MATCH (a:Person)-[:ACTED_IN]->(m:Movie)<-[:DIRECTED]-(d:Person) +WHERE m.year > 2000 +RETURN a.name, m.title, d.name +``` + +### Optionally matching rows + +``` +MATCH (m:Movie) WHERE m.title = "Kiss Me Deadly" +MATCH (m)-[:IN_GENRE]->(g:Genre)<-[:IN_GENRE]-(rec:Movie) +OPTIONAL MATCH (m)<-[:ACTED_IN]-(a:Actor)-[:ACTED_IN]->(rec) +RETURN rec.title, a.name +``` + +This query returns rows where the pattern where an actor acted in both movies is optional and a null value is returned for any row that has no value. In general, and depending on your graph, an optional match will return more rows. + +## Controlling Results Returned + +### Ordering Returned Results + +``` +MATCH (p:Person) +WHERE p.born.year = 1980 +RETURN p.name AS name, p.born AS birthDate +ORDER BY birthDate DESC , name ASC +``` + +### Limiting results; Skipping some results + +``` +MATCH (p:Person) +WHERE p.born.year = 1980 +RETURN p.name as name, +p.born AS birthDate +ORDER BY p.born SKIP 40 LIMIT 10 +``` + +In this query, we return 10 rows representing page 5, where each page contains 10 rows. + +``` +MATCH (p:Person)-[:ACTED_IN| DIRECTED]->(m) +WHERE m.title = 'Toy Story' +MATCH (p)-[:ACTED_IN]->()<-[:ACTED_IN]-(p2:Person) +RETURN p.name, p2.name +``` + +Returns the names people who acted or directed the movie Toy Story and then retrieves all people who acted in the same movie. + +### Map projections + +``` +MATCH (p:Person) +WHERE p.name CONTAINS "Thomas" +RETURN p { .* } AS person +ORDER BY p.name ASC +``` + +``` +MATCH (p:Person) +WHERE p.name CONTAINS "Thomas" +RETURN p { .name, .born } AS person +ORDER BY p.name +``` + +``` +MATCH (m:Movie)<-[:DIRECTED]-(d:Director) +WHERE d.name = 'Woody Allen' +RETURN m {.*, favorite: true} AS movie +``` + +Returning a property of favorite with a value of true for each Movie object returned. + +``` +MATCH (m:Movie)<-[:ACTED_IN]-(p:Person) +WHERE p.name = 'Henry Fonda' +RETURN m.title AS movie, +CASE +WHEN m.year < 1940 THEN 'oldies' +WHEN 1940 <= m.year < 1950 THEN 'forties' +WHEN 1950 <= m.year < 1960 THEN 'fifties' +WHEN 1960 <= m.year < 1970 THEN 'sixties' +WHEN 1970 <= m.year < 1980 THEN 'seventies' +WHEN 1980 <= m.year < 1990 THEN 'eighties' +WHEN 1990 <= m.year < 2000 THEN 'nineties' +ELSE 'two-thousands' +END +AS timeFrame +``` + +# Aggregating Data + +If a aggregation function like count() is used, all non-aggregated result columns become grouping keys. + +_If you specify **count(n)**, the graph engine calculates the number of non-null occurrences of n. +If you specify \*\*count(_)\*_, the graph engine calculates the number of rows retrieved, including those with null values._ + +### Returning a list + +``` +MATCH (p:Person) +RETURN p.name, [p.born, p.died] AS lifeTime +LIMIT 10 +``` + +``` +MATCH (a:Person)-[:ACTED_IN]->(m:Movie) +WHERE m.year = 1920 +RETURN collect( DISTINCT m.title) AS movies, +collect( a.name) AS actors +``` + +``` +MATCH (a:Person)-[:ACTED_IN]->(m:Movie) +RETURN m.title AS movie, +collect(a.name)[2..] AS castMember, +size(collect(a.name)) as castSize +``` + +return a slice of a collection. + +### List comprehension + +``` +MATCH (m:Movie) +RETURN m.title as movie, +[x IN m.countries WHERE x = 'USA' OR x = 'Germany'] +AS country LIMIT 500 +``` + +### Pattern comprehension + +``` +MATCH (m:Movie) +WHERE m.year = 2015 +RETURN m.title, +[(dir:Person)-[:DIRECTED]->(m) | dir.name] AS directors, +[(actor:Person)-[:ACTED_IN]->(m) | actor.name] AS actors +``` + +For pattern comprehension specify the list with the square braces to include the pattern followed by the pipe character to then specify what value will be placed in the list from the pattern. + +``` +[ | value] + +``` + +MATCH (a:Person {name: 'Tom Hanks'}) +RETURN [(a)-->(b:Movie) +WHERE b.title CONTAINS "Toy" | b.title + ": " + b.year] +AS movies + +``` + +### Working with maps + +A Cypher map is list of key/value pairs where each element of the list is of the format 'key': value. + +``` + +RETURN {Jan: 31, Feb: 28, Mar: 31, Apr: 30 , +May: 31, Jun: 30 , Jul: 31, Aug: 31, Sep: 30, +Oct: 31, Nov: 30, Dec: 31}['Feb'] AS daysInFeb + +Also with dot notation Dec: 31}.Feb AS daysInFeb + +### Map projections + +``` +MATCH (m:Movie) +WHERE m.title CONTAINS 'Matrix' +RETURN m { .title, .released } AS movie +``` + +# Working with Dates and Times + +``` +RETURN date(), datetime(), time() + +``` + +``` + +CALL apoc.meta.nodeTypeProperties() + +``` + +List node properties + +``` + +MATCH (x:Test {id: 1}) +RETURN x.date.day, x.date.year, +x.datetime.year, x.datetime.hour, +x.datetime.minute + +``` + +Extract date components + +``` + +MATCH (x:Test {id: 1}) +SET x.datetime1 = datetime('2022-01-04T10:05:20'), +x.datetime2 = datetime('2022-04-09T18:33:05') +RETURN x + +``` + +`Date property using a string.` + +``` +MATCH (x:Test {id: 1}) +RETURN duration.between(x.date1,x.date2) +RETURN duration.inDays(x.datetime1,x.datetime2).days +RETURN x.date1 + duration({months: 6}) +``` + +### APOC to format dates and times + +``` +MATCH (x:Test {id: 1}) +RETURN x.datetime as Datetime, +apoc.temporal.format( x.datetime, 'HH:mm:ss.SSSS') +AS formattedDateTime + +``` + +# Graph Traversal + +### Anchor of a query + +Execution plan determines the set of nodes, which are the starting points for the query. The anchor is ostly based on the match clause. +The anchor is typically determined by meta-data that is stored in the graph or a filter that is provided inline or in a WHERE clause. The anchor for a query will be based upon the fewest number of nodes that need to be retrieved into memory. + +# Varying Length Traversal + +``` +MATCH p = shortestPath((p1:Person)-[*]-(p2:Person)) +WHERE p1.name = "Eminem" +AND p2.name = "Charlton Heston" +RETURN p +``` + +shortest path, regardless of relations + +``` +MATCH (p:Person {name: 'Eminem'})-[:ACTED_IN*2]-(others:Person) +RETURN others.name + +``` + +Two hops away from Eminem using the ACTED_IN relationship + +``` +MATCH (p:Person {name: 'Eminem'})-[:ACTED_IN*1..4]-(others:Person) +RETURN others.name +``` + +1 to 4 nodes; all connections of the connectod nodes; 4 deep + +# Pipelining Queries + +``` +MATCH (n:Movie) +WHERE n.imdbRating IS NOT NULL +AND n.poster IS NOT NULL +WITH n { + .title, + .year, + .languages, + .plot, + .poster, + .imdbRating, + directors: [ (n)<-[:DIRECTED]-(d) | d { tmdbId:d.imdbId, .name } ] +} +ORDER BY n.imdbRating DESC LIMIT 4 +RETURN collect(n) +``` + +``` +WITH 'Clint Eastwood' AS a, 'high' AS t +MATCH (p:Person)-[:ACTED_IN]->(m:Movie) +with p, m, toLower(m.title) as movieTitle +WHERE p.name = a +AND movieTitle CONTAINS t +RETURN p.name AS actor, m.title AS movie +``` + +``` +WITH 'Tom Hanks' AS theActor +MATCH (p:Person)-[:ACTED_IN]->(m:Movie) +WHERE p.name = theActor +AND m.revenue IS NOT NULL +with m order by m.revenue desc limit 1 +// Use WITH here to limit the movie node to 1 and order it by revenue +RETURN m.revenue AS revenue, m.title AS title +``` + +``` +MATCH (n:Movie) +WHERE n.imdbRating IS NOT NULL and n.poster IS NOT NULL +with n { + .title, + .imdbRating, + actors: [(a)-[:ACTED_IN]->(n) | a {name:a.name, .name}], + genre: [(n)-[:IN_GENRE]->(g) | g {name:g.name, .name}]} +ORDER BY n.imdbRating DESC LIMIT 4 +with collect(n.actors) as a +unwind a as b +unwind b as listB +return listB.name, count(listB.name) +order by listB.name +``` + +# Pipelining Queries + +### Aggregation and pipelining + +``` +MATCH (:Movie {title: 'Toy Story'})-[:IN_GENRE]->(g:Genre)<-[:IN_GENRE]-(m) +WHERE m.imdbRating IS NOT NULL +WITH + g.name AS genre, + count(m) AS moviesInCommon, + sum(m.imdbRating) AS total +RETURN + genre, moviesInCommon, + total/moviesInCommon AS score +ORDER By score DESC +``` + +``` +MATCH (u:User {name: "Misty Williams"})-[r:RATED]->(:Movie) +WITH u, avg(r.rating) AS average +MATCH (u)-[r:RATED]->(m:Movie) +WHERE r.rating > average +RETURN + average , m.title AS movie, + r.rating as rating +ORDER BY rating DESC +``` + +### Using WITH for collecting + +``` +MATCH (m:Movie)--(a:Actor) +WHERE m.title CONTAINS 'New York' +WITH + m, + collect (a.name) AS actors, + count(*) AS numActors +RETURN + m.title AS movieTitle, + actors +ORDER BY numActors DESC +``` + +``` +MATCH (m:Movie)<-[:ACTED_IN]-(a:Actor) +WHERE m.title CONTAINS 'New York' +WITH + m, + collect (a.name) AS actors, + count(*) AS numActors +ORDER BY numActors DESC +RETURN collect(m { .title, actors, numActors }) AS movies +``` + +### Using LIMIT early + +``` +MATCH (p:Actor) +WHERE p.born.year = 1980 +WITH p LIMIT 3 +MATCH (p)-[:ACTED_IN]->(m:Movie)-[:IN_GENRE]->(g:Genre) +WITH + p, + collect(DISTINCT g.name) AS genres +RETURN p.name AS actor, genres +``` + +``` +Match (a:Actor)-[:ACTED_IN]->(m) +where a.name = 'Tom Hanks' +with m +match (m)<-[r:RATED]-(u) +with + m, + avg(r.rating) as rating +return rating, m.title +order by rating desc +limit 1 +``` + +# Unwinding Lists + +``` +MATCH (m:Movie) +UNWIND m.languages AS lang +WITH + m, + trim(lang) AS language +// this automatically, makes the language distinct because it's a grouping key +WITH + language, + collect(m.title) AS movies +RETURN + language, + movies[0..10] +``` + +# Reducing Memory (CALL, UNION) + +MATCH clauses exceed the VM configured, the query will fail. +A subquery is a set of Cypher statements that execute within their own scope. + +Important things to know about a subquery: + +- A subquery returns values referred to by the variables in the RETURN clause. +- A subquery cannot return variables with the same name used in the enclosing query. +- You must explicitly pass in variables from the enclosing query to a subquery. + +### CALL + +``` +MATCH (m:Movie) +CALL { + WITH m + MATCH (m)<-[r:RATED]-(u:User) + WHERE r.rating = 5 + RETURN count(u) AS numReviews +} +RETURN m.title, numReviews +ORDER BY numReviews DESC +``` + +### UNION [ALL] + +``` +MATCH (p:Person) +WITH p LIMIT 100 +CALL { + WITH p + OPTIONAL MATCH (p)-[:ACTED_IN]->(m:Movie) + RETURN m.title + ": " + "Actor" AS work +UNION + WITH p + OPTIONAL MATCH (p)-[:DIRECTED]->(m:Movie) + RETURN m.title+ ": " + "Director" AS work +} +RETURN p.name, collect(work) +``` + +``` +MATCH (g:Genre) +call { + with g + match (m:Movie)-[:IN_GENRE]->(g) + where 'France' in m.countries + return count(m) as numMovies +} +RETURN g.name AS genre, numMovies +ORDER BY numMovies DESC +``` + +# Using Parameters + +``` +:params {actorName: 'Tom Cruise', movieName: 'Top Gun'} +``` + +``` +MATCH (p:Person)-[:ACTED_IN]->(m:Movie) +WHERE p.name = $actorName +RETURN m.released AS releaseDate, +m.title AS title +ORDER BY m.released DESC +``` + +``` +:params {actorName: 'Tom Cruise', movieName: 'Top Gun', l:2} + +MATCH (p:Person)-[:ACTED_IN]->(m:Movie) +WHERE m.title = $movieName RETURN p.name LIMIT $l +``` + +### Setting Integers + +:param number: 10 >>>> will be converted to float!!!!! +:param number=> 10 >>>>> remains an integer!!!! + +``` +:param +``` + +to view all set parameters + +``` +:param {} +``` + +clear all set parameters + +# Application Examples Using Parameters + +``` +def get_actors(tx, movieTitle): # (1) + result = tx.run(""" + MATCH (p:Person)-[:ACTED_IN]->(m:Movie) + WHERE m.title = $title + RETURN p + """, title=movieTitle) + + # Access the `p` value from each record + return [ record["p"] for record in result ] + +with driver.session() as session: + result = session.read_transaction(get_actors, movieTitle="Toy Story") +``` diff --git a/Databases/Neo4j/Preparing for Importing Data.md b/Databases/Neo4j/Preparing for Importing Data.md new file mode 100644 index 0000000..3523f31 --- /dev/null +++ b/Databases/Neo4j/Preparing for Importing Data.md @@ -0,0 +1,87 @@ +--- +title: 'Preparing for Importing Data ' +updated: 2022-07-31 11:38:21Z +created: 2022-07-30 13:45:09Z +--- + +# What does importing data mean? + +Cypher has a built-in clause, **LOAD CSV** for importing CSV files. If you have a JSON or XML file, you must use the **APOC library** to import the data, but you can also import CSV with APOC. And the **Neo4j Data Importer** enables you to import CSV data without writing any Cypher code. + +The **types of data** that you can store as properties in Neo4j include: + +- String +- Long (integer values) +- Double (decimal values) +- Boolean +- Date/Datetime +- Point (spatial) +- StringArray (comma-separated list of strings) +- LongArray (comma-separated list of integer values) +- DoubleArray (comma-separated list of decimal values) + +### Two ways that you can import CSV data: + +1. Using the Neo4j Data Importer. +2. Writing Cypher code to perform the import. + +### Steps for preparing for importing data + +1. Understand the data in the source CSV files. +2. Inspect and clean (if necessary) the data in the source data files. +3. Create or understand the graph data model you will be implementing during the import. + +# Understanding the Source Data + +CSV files, you must determine: + +- Whether the CSV file will have header information, describing the names of the fields. +- What the delimiter will be for the fields in each row. + +Including headers in the CSV file **reduces syncing** issues and is a recommended Neo4j best practice. + +A Neo4j best practice is to **use an ID as a unique property value for each node**. If the IDs in your CSV file are not unique for the same entity (node), you will have problems when you load the data and try to create relationships between existing nodes. + +### Inspecting the Data for Import + +**Important**: By default all of these fields in each row will be read in as string types. +Use **FIELDTERMINATOR** is delimiter is not the default ',' + +Test if all rows in the csv file can be read. For example: + +``` +LOAD CSV WITH HEADERS +FROM 'https://data.neo4j.com/importing/ratings.csv' +AS row +RETURN count(row) +``` + +### Is the data clean? + +Check: + +- Are quotes used correctly? +- If an element has no value will an empty string be used? +- Are UTF-8 prefixes used (for example \uc)? +- Do some fields have trailing spaces? +- Do the fields contain binary zeros? +- Understand how lists are formed (default is to use colon(:) as the separator. +- Any obvious typos? + +# Overview of the Neo4j Data Importer + +The benefit of the Data Importer is that you need not know Cypher to load the data. +It is useful for loading small to medium CSV files that contain fewer that 1M rows. +Data that is imported into the graph can be interpreted as string, integer, float, or boolean data. + +Requirements for using the Data Importer +- You must use CSV files for import. +- CSV files must reside on your local system so you can load them into the graph app. +- CSV data must be clean. +- IDs must be unique for all nodes you will be creating. +- The CSV file must have headers. +- The DBMS must be started. + +If you have de-normalized data, you will need to perform a multi-pass import. That is, you cannot create multiple nodes and relationship types from a single CSV file. + +The Neo4j Data Importer can import or export mappings to a JSON file or to a ZIP file, if you also want to include the CSV files. \ No newline at end of file diff --git a/Databases/Neo4j/Refactoring Imported Data.md b/Databases/Neo4j/Refactoring Imported Data.md new file mode 100644 index 0000000..4d233f7 --- /dev/null +++ b/Databases/Neo4j/Refactoring Imported Data.md @@ -0,0 +1,121 @@ +--- +title: Refactoring Imported Data +updated: 2022-07-31 20:32:04Z +created: 2022-07-31 17:34:56Z +--- + +# Transforming String Properties to Dates + +### Converting to Date values + +date(property) + +- correct data format eq "yyyy-mm-dd" +- not empty + [Cypher: Temporal (Date/Time) values](https://neo4j.com/docs/cypher-manual/current/syntax/temporal/) + +Test for empty string + +``` +MATCH (p:Person) +SET p.born = CASE p.born WHEN "" THEN null ELSE date(p.born) END +WITH p +SET p.died = CASE p.died WHEN "" THEN null ELSE date(p.died) END +``` + +List all stored **node** types in the database: + +``` +CALL apoc.meta.nodeTypeProperties() +``` + +List all stored **relation** types in the database: + +``` +CALL apoc.meta.relTypeProperties() +``` + +# Transforming Multi-value Properties + +### Transform Strings to Lists + +- list +- same type + +Transform to list from string with seperator eq "|" + +``` +MATCH (m:Movie) +SET m.countries = split(coalesce(m.countries,""), "|") +``` + +Transform a multi-value property to a list of strings => StringArray in database + +# Adding labels + +Labels is a best practice so that key queries will perform better, especially when the graph is large. + +``` +MATCH (p:Person)-[:ACTED_IN]->() +WITH DISTINCT p SET p:Actor +``` + +# Refactoring Properties as Nodes + +Increase performance +For unique properties, like id's, create indexes. +Best practice is to always have a unique ID for every type of node in the graph. +View defined constraints in database + +``` +SHOW CONSTRAINTS +``` + +Before using MERGE, create first a unique constaint + +``` +CREATE CONSTRAINT Genre_name ON (g:Genre) ASSERT g.name IS UNIQUE +CREATE CONSTRAINT Genre_name IF NOT EXISTS ON (x:Genre) ASSERT x.name IS UNIQUE +``` + +Creating the new nodes from a node property + +``` +MATCH (m:Movie) +UNWIND m.genres AS genre +WITH m, genre +MERGE (g:Genre {name:genre}) +MERGE (m)-[:IN_GENRE]->(g) +``` + +eq + +``` +unwind ['aap','olifant'] as a +return a +``` + +Removing a node property, set it to NULL + +``` +MATCH (m:Movie) +SET m.genres = null +``` + +Show the schema of a database + +``` +CALL db.schema.visualization +``` + +# Importing Large Datasets with Cypher + +Data Importer can be used for small to medium datasets containing less than 1M rows + +In Cypher, by default, the execution of your code is a single transaction. Break up the execution of the Cypher into multiple transactions. Reduces the amount of memory needed for the import. In Neo4j: + +``` +:auto USING PERIODIC COMMIT LOAD CSV .... +``` + +The advantage of performing the import in multiple passes is that you can check the graph after each import to see if it is getting closer to the data model. If the CSV file were extremely large, you might want to consider a single pass. diff --git a/Databases/Neo4j/Syntax_examples.md b/Databases/Neo4j/Syntax_examples.md new file mode 100644 index 0000000..8259217 --- /dev/null +++ b/Databases/Neo4j/Syntax_examples.md @@ -0,0 +1,343 @@ +--- +title: Syntax_examples +updated: 2022-07-18 09:37:40Z +created: 2021-05-04 14:58:11Z +--- + +## show database schema info +CALL db.schema.visualization() +CALL db.schema.relTypeProperties() +CALL db.schema.nodeTypeProperties() +CALL db.propertyKeys() + +## syntax +MATCH (variable:Label {propertyKey: propertyValue, propertyKey2: propertyValue2}) +RETURN variable + +## relationships +() // a node +()--() // 2 nodes have some type of relationship +()-[]-() // 2 nodes have some type of relationship +()-->() // the first node has a relationship to the second node +()<--() // the second node has a relationship to the first node + +MATCH (node1)-[:REL_TYPE]->(node2) +RETURN node1, node2 + +MATCH (node1)-[:REL_TYPEA | REL_TYPEB]->(node2) +RETURN node1, node2 + + + +## show node with name "Tom Hanks" +MATCH (tom {name: "Tom"}) RETURN tom + +## return all nodes in database +MATCH (a:Person) WHERE a.name = "Tom" RETURN a +MATCH (a:Person) RETURN a.name + +## with where clause +match (a:Movie) +where a.released >= 1990 and a.released < 1999 +return a.title; + +## a list of all properties that match a string +MATCH (n) WITH keys(n) AS p UNWIND p AS x WITH DISTINCT x WHERE x =~ ".*" RETURN collect(x) AS SET; + +## delete all nodes and relations +MATCH (n) +DETACH DELETE n + +## create +```cypher +create (:Person {name = 'jan', age = 32}) +``` + +match(n:Person {age: 32}) return n + +match(n:Person {age: 32}) +create (n)-[:RELATIE]->(:Person {name:"klaas"}) + +MATCH (n:Person) +DETACH DELETE n + +MATCH (p:Person)-[:ACTED_IN]->(m:Movie)<-[:DIRECTED]-(other:Person) +where toLower(p.name) =~ 'gene.*' and other.born IN [1950,1930] + and exists((other)-[:DIRECTED]->(m)) +return m.title, other.name, other.born as YearBorn + +MATCH (p:Person)-[:ACTED_IN]->(m:Movie) +where p.name = 'Tom Hanks' +with m, datetime().year - m.released as Ago, m.released - p.born as Age +where 20 <= Ago <= 33 +return m.title, Ago + +MATCH (m:Movie) +WITH m, size((:Person)-[:DIRECTED]->(m)) AS directors +WHERE directors >= 2 +OPTIONAL MATCH (p:Person)-[:REVIEWED]->(m) +RETURN m.title, p.name + + +match (a:Person), (m:Movie), (b:Person) +where a.name = 'Liam Neeson' + and b.name = 'Benjamin Melniker' + and m.title = 'Batman Begins' + create (a)-[:ACTED_IN {roles: ['Rachel','Rachel Dawes']}]->(m)<-[:PRODUCED]-(b) + return a,m,b + +MATCH (a:Person),(m:Movie) +WHERE a.name = 'Christian Bale' AND + m.title = 'Batman Begins' AND + NOT exists((a)-[:ACTED_IN]->(m)) +CREATE (a)-[rel:ACTED_IN]->(m) +SET rel.roles = ['Bruce Wayne','Batman'] +RETURN a, rel, m + +MATCH (p:Person)-[rel:ACTED_IN]->(m:Movie) +where m.title = 'Forrest Gump' +set rel.roles = case p.name +when 'Tom Hanks' then ['Forrest Gum'] +when 'Robin Wright' then ['Jenny Curran'] +when 'Gary Sinise' then ['Lieutenant Dan Taylor'] +end +return p,rel,m + +MATCH (p:Person)-[rel:HELPED]->(p2:Person) +where p.name = 'Tom Hanks' and p2.name = 'Gary Sinise' +set rel += {research:'war history'} + return p,rel,p2 + +merge (m:Movie {name:'Forrest Gump'}) +on match set m.year = 1994 +on match set m.tagline = 'Life is like a box of chocolates…​you never know what you’re gonna get.' +return m + +merge (p:Movie {name:'Forrest Gump'}) +on match set p:OlderMovie +return p + +match (p:Person {name:'Robert Zemeckis'}), (m:Movie {title:'Forrest Gump'}) +merge (p)-[r:DIRECTED]->(m) +return p,r,m + + +## constrain uniqueness +CREATE CONSTRAINT UniqueMovieTitleConstraint + ON (m:Movie) + ASSERT m.title IS UNIQUE + +## constrain uniqueness over two properties +## only enterprise edition +CREATE CONSTRAINT UniqueNameBornConstraint + ON (p:Person) + ASSERT (p.name, p.born) IS NODE KEY + +## needs enterprise edition of neo4j +create constraint PersonBornExistsConstraint on (p:Person) +assert exists(p.born) + + +## existence constraint (possible for node +CREATE CONSTRAINT ExistsMovieTagline + ON (m:Movie) + ASSERT exists(m.tagline) + +DROP CONSTRAINT MovieTitleConstraint + +## existence constraint for relationship +## only enterprise edition of neo4j +CREATE CONSTRAINT ExistsREVIEWEDRating + ON ()-[rel:REVIEWED]-() + ASSERT exists(rel.rating) + +## drop constraint +DROP CONSTRAINT ExistsREVIEWEDRating + +CALL db.constraints() better SHOW CONSTRAINTS + +## Indexes +## Single property index +CREATE INDEX MovieReleased FOR (m:Movie) ON (m.released) + +## composite index +CREATE INDEX MovieReleasedVideoFormat + FOR (m:Movie) + ON (m.released, m.videoFormat) + +## full-text schema index +CALL db.index.fulltext.createNodeIndex( + 'MovieTitlePersonName',['Movie', 'Person'], ['title', 'name']) +### To use a full-text schema index, you must call the query procedure that uses the index. +CALL db.index.fulltext.queryNodes( + 'MovieTitlePersonName', 'Jerry') + YIELD node, score + RETURN node.title, score + +### Searching on a particular property +CALL db.index.fulltext.queryNodes( + 'MovieTitlePersonName', 'name: Jerry') YIELD node +RETURN node + +## drop index +DROP INDEX MovieReleasedVideoFormat + +## dropping full-text schema index +CALL db.index.fulltext.drop('MovieTitlePersonName') + +## search a full-text schema index +CALL db.index.fulltext.queryNodes('MovieTaglineFTIndex', 'real OR world') + YIELD node + RETURN node.title, node.tagline + +## set parameters +:param year => 2000 +:params {actorName: 'Tom Cruise', movieName: 'Top Gun'} +## for statement +MATCH (p:Person)-[:ACTED_IN]->(m:Movie) +WHERE p.name = $actorName AND m.title = $movieName +RETURN p, m +## clear +:params {} +## view +:params + +## Analyzing queries +- EXPLAIN provides estimates of the graph engine processing that will occur, but does not execute the Cypher statement. + +- PROFILE provides real profiling information for what has occurred in the graph engine during the query and executes the Cypher statement. (run-time performance metrics) + +## Monitoring queries +:queries + + +## exercise +:params {year:2006, ratingValue:65} + +match (p:Person)-[r:REVIEWED]->(m:Movie)<-[:ACTED_IN]-(a:Person) +where m.released = $year and r.rating = $ratingValue +return p.name, m.title, m.released, r.rating, collect(a.name) + + +:auto USING PERIODIC COMMIT LOAD CSV +commit every 1000 rows +Eager operators don't act on this command, ie: + collect() + count() + ORDER BY + DISTINCT + +LOAD CSV WITH HEADERS FROM +'https://data.neo4j.com/v4.0-intro-neo4j/directors.csv' AS row +MATCH (movie:Movie {id:toInteger(row.movieId)}) +MATCH (person:Person {id: toInteger(row.personId)}) +MERGE (person)-[:DIRECTED]->(movie) +ON CREATE SET person:Director + +LOAD CSV WITH HEADERS +FROM 'http://data.neo4j.com/v4.0-intro-neo4j/actors.csv' +AS line +MERGE (actor:Person {name: line.name}) + ON CREATE SET actor.born = toInteger(trim(line.birthYear)), actor.actorId = line.id + ON MATCH SET actor.actorId = line.id + + + +## before load +CREATE CONSTRAINT UniqueMovieIdConstraint ON (m:Movie) ASSERT m.id IS UNIQUE; +## after load +CREATE INDEX MovieTitleIndex ON (m:Movie) FOR (m.title); + + + +// Delete all constraints and indexes +CALL apoc.schema.assert({},{},true); +// Delete all nodes and relationships +CALL apoc.periodic.iterate( + 'MATCH (n) RETURN n', + 'DETACH DELETE n', + { batchSize:500 } +) + + +## test apoc +CALL dbms.procedures() +YIELD name WHERE name STARTS WITH "apoc" +RETURN name + + +## Graph modelling +How does Neo4j support graph data modeling? +- allows you to create property graphs. +- traversing the graph: traversal means anchoring a query based upon a property value, then traversing the graph to satisfy the query + + +Nodes and relationships are the key components of a graph. +Nodes must have labels to categorize entities. +A label is used to categorize a set of nodes. +Relationships must have direction and type. +A relationship is only traversed once during a query. +Nodes and relationships can have properties. +Properties are used to provide specific values to a node or relationship. + +## Your model must address Nodes: +- Uniqueness of nodes: always have a property (or set of properties) that uniquely identify a node. +- Complex data: balance between number of properties that represent complex data vs. multiple nodes and relationships. + +super nodes = (a node with lots of fan-in or fan-out) +- Reduce property duplication (no repeating property values) +- Reduce gather-and-inspect (traversal) + +## Best practices for modeling relationships +- Using specific relationship types. +- Reducing symmetric relationships. + - No semantically identical relationships (PARENT_OF and CHILD_OF) + - Not all mutual relationships are semantically symmetric(FOLLOWS) +- Using types vs. properties. + +## Property best practices +In the case of property value complexity, it depends on how the property is used. Anchors and traversal paths that use property values need to be parsed at query time. + +- Property lookups have a cost. +- Parsing a complex property adds more cost. +- Anchors and properties used for traversal will be as simple as possible. +- Identifiers, outputs, and decoration are OK as complex values. + +## Hierarchy of accessibility +1. Anchor node label, indexed anchor node properties (cheap) +2. Relationship types (cheap) +3. Non-indexed anchor node properties +4. Downstream node labels +5. Relationship properties, downstream node properties + +Downstream labels and properties are most expensive. + +## Common graph structures used in modeling: +1. Intermediate nodes + - (solve hyperedge; n-ary relationships) + - sharing context (share contextual information) + - sharing data (deduplicate information) + - organizing data (avoid density of nodes) +2. Linked lists (useful whenever the sequence of objects matters) + - Interleaved linked list + - Head and tail of linked list (root point to head and tail) + - No double linked-lists (redundant symmetrical relationships) +3. Timeline trees + - use time as either an anchor or a navigational aid + - topmost node in the timeline is an “all time” node + - ​timeline trees consume a lot of space +4. Multiple structures in a single graph + + +CREATE (:Airport {code: "ABQ"})<-[:CONNECTED_TO {airline: "WN", flightNumber: 500, date: "2019-1-3", depature: 1445, arrival: 1710}]-(:Airport {code: "LAS"})-[:CONNECTED_TO {airline: "WN", flightNumber: 82, date: "2019-1-3", depature: 1715, arrival: 1820}]->(:Airport {code: "LAX"}) + + +LOAD CSV WITH HEADERS FROM 'file:///flights_2019_1k.csv' AS row +MERGE (origin:Airport {code: row.Origin}) +MERGE (destination:Airport {code: row.Dest}) +MERGE (origin)-[connection:CONNECTED_TO { + airline: row.UniqueCarrier, + flightNumber: row.FlightNum, + date: toInteger(row.Year) + '-' + toInteger(row.Month) + '-' + toInteger(row.DayofMonth)}]->(destination) +ON CREATE SET connection.departure = toInteger(row.CRSDepTime), connection.arrival = toInteger(row.CRSArrTime) + diff --git a/Databases/SQL vs NoSQL_ 5 Critical Differences.md b/Databases/SQL vs NoSQL_ 5 Critical Differences.md new file mode 100644 index 0000000..62744de --- /dev/null +++ b/Databases/SQL vs NoSQL_ 5 Critical Differences.md @@ -0,0 +1,17 @@ +--- +title: 'SQL vs NoSQL: 5 Critical Differences' +updated: 2022-05-24 19:33:07Z +created: 2022-05-24 19:31:42Z +--- + +# SQL vs NoSQL: 5 Critical Differences + +The five critical differences between SQL vs NoSQL are: +1. SQL databases are relational, NoSQL databases are non-relational. +2. SQL databases use structured query language and have a predefined schema. NoSQL databases have dynamic schemas for unstructured data. +3. SQL databases are vertically scalable, while NoSQL databases are horizontally scalable. +4. SQL databases are table-based, while NoSQL databases are document, key-value, graph, or wide-column stores. +5. SQL databases are better for multi-row transactions, while NoSQL is better for unstructured data like documents or JSON. + +[source](https://www.integrate.io/blog/the-sql-vs-nosql-difference/) + diff --git a/DeepLearning_AI/Course 1_ Neural Networks and Deep Learning.md b/DeepLearning_AI/Course 1_ Neural Networks and Deep Learning.md new file mode 100644 index 0000000..7d47f87 --- /dev/null +++ b/DeepLearning_AI/Course 1_ Neural Networks and Deep Learning.md @@ -0,0 +1,635 @@ +--- +title: 'Course 1: Neural Networks and Deep Learning' +updated: 2022-05-22 11:18:21Z +created: 2022-05-16 17:52:57Z +--- + +# Course 1: Neural Networks and Deep Learning + +- [Course 1: Neural Networks and Deep Learning](#course-1-neural-networks-and-deep-learning) + - [Week 1: Introduction to Deep Learning](#week-1-introduction-to-deep-learning) + - [Learning Objectives](#learning-objectives) + - [Introduction to Deep Learning](#introduction-to-deep-learning) + - [What is a neural network](#what-is-a-neural-network) + - [Supervised learning with neural networks](#supervised-learning-with-neural-networks) + - [Why is deep learning taking off](#why-is-deep-learning-taking-off) + - [Week 2: Neural Networks Basics](#week-2-neural-networks-basics) + - [Learning Objectives](#learning-objectives-1) + - [Logistic Regression as a Neural Network](#logistic-regression-as-a-neural-network) + - [Binary Classification](#binary-classification) + - [Logistic Regression](#logistic-regression) + - [Logistic Regression Cost Function](#logistic-regression-cost-function) + - [Gradient Descent](#gradient-descent) + - [Derivatives](#derivatives) + - [Computation Graph](#computation-graph) + - [Derivatives with a Computation Graph](#derivatives-with-a-computation-graph) + - [Logistic Regression Gradient Descent](#logistic-regression-gradient-descent) + - [Gradient Descent on m Examples](#gradient-descent-on-m-examples) + - [Derivation of dL/dz](#derivation-of-dldz) + - [Python and Vectorization](#python-and-vectorization) + - [Vectorization](#vectorization) + - [Vectorizing Logistic Regression](#vectorizing-logistic-regression) + - [Broadcasting in Python](#broadcasting-in-python) + - [A note on python/numpy vectors](#a-note-on-pythonnumpy-vectors) + - [Quick tour of Jupyter/iPython Notebooks](#quick-tour-of-jupyteripython-notebooks) + - [Explanation of logistic regression cost function (optional)](#explanation-of-logistic-regression-cost-function-optional) + - [Week 3: Shallow Neural Networks](#week-3-shallow-neural-networks) + - [Learning Objectives](#learning-objectives-2) + - [Shallow Neural Network](#shallow-neural-network) + - [Neural Networks Overview](#neural-networks-overview) + - [Neural Network Representation](#neural-network-representation) + - [Computing a Neural Network's Output](#computing-a-neural-networks-output) + - [Vectorizing across multiple examples](#vectorizing-across-multiple-examples) + - [Activation functions](#activation-functions) + - [Why do you need non-linear activation functions](#why-do-you-need-non-linear-activation-functions) + - [Derivatives of activation functions](#derivatives-of-activation-functions) + - [Gradient descent for Neural Networks](#gradient-descent-for-neural-networks) + - [Random initialization](#random-initialization) + - [Week 4: Deep Neural Networks](#week-4-deep-neural-networks) + - [Learning Objectives](#learning-objectives-3) + - [Deep Neural Network](#deep-neural-network) + - [Deep L-layer neural network](#deep-l-layer-neural-network) + - [Forward Propagation in a deep network](#forward-propagation-in-a-deep-network) + - [Getting your matrix dimensions right](#getting-your-matrix-dimensions-right) + - [Why deep representations](#why-deep-representations) + - [Building blocks of deep neural networks](#building-blocks-of-deep-neural-networks) + - [Forward and Backward Propagation](#forward-and-backward-propagation) + - [Parameters vs Hyperparameters](#parameters-vs-hyperparameters) + - [What does this have to do with the brain](#what-does-this-have-to-do-with-the-brain) + +## Week 1: Introduction to Deep Learning + +### Learning Objectives + +- Discuss the major trends driving the rise of deep learning +- Explain how deep learning is applied to supervised learning +- List the major categories of models (CNNs, RNNs, etc.), and when they should be applied +- Assess appropriate use cases for deep learning + +### Introduction to Deep Learning + +#### What is a neural network + +It is a powerful learning algorithm inspired by how the brain works. Here is a definition from [mathworks](https://www.mathworks.com/discovery/neural-network.html): + +![neural-network](../_resources/neural-network.svg "Neural Network") + +*image source: [mathworks](https://www.mathworks.com/discovery/neural-network.html)* + +> A neural network (also called an artificial neural network) is an adaptive system that learns by using interconnected nodes or neurons in a layered structure that resembles a human brain. A neural network can learn from data—so it can be trained to recognize patterns, classify data, and forecast future events. +> +> A neural network breaks down the input into layers of abstraction. It can be trained using many examples to recognize patterns in speech or images, for example, just as the human brain does. Its behavior is defined by the way its individual elements are connected and by the strength, or weights, of those connections. These weights are automatically adjusted during training according to a specified learning rule until the artificial neural network performs the desired task correctly. +> +> A neural network combines several processing layers, using simple elements operating in parallel and inspired by biological nervous systems. It consists of an input layer, one or more hidden layers, and an output layer. In each layer there are several nodes, or neurons, with each layer using the output of the previous layer as its input, so neurons interconnect the different layers. Each neuron typically has weights that are adjusted during the learning process, and as the weight decreases or increases, it changes the strength of the signal of that neuron. + +#### Supervised learning with neural networks + +In supervised learning, we are given a data set and already know what our correct output should look like, having the idea that there is a relationship between the input and the output. + +Supervised learning problems are categorized into "regression" and "classification" problems. In a regression problem, we are trying to predict results within a continuous output, meaning that we are trying to map input variables to some continuous function. In a classification problem, we are instead trying to predict results in a discrete output. In other words, we are trying to map input variables into discrete categories. + +*Examples of supervised learning applications*: + +| Input(X) | Output(y) | Application | +| --- | --- | --- | +| Home features | Price | Real Estate | +| Ad, user info | Click on ad? (0/1) | Online Advertising | +| Image | Object (1,...,1000) | Photo tagging | +| Audio | Text transcript | Speech recognition | +| English | Chinese | Machine translation | +| Image, Radar info | Position of other cars | Autonomous driving | + +*Structured vs unstructured data* + +- Structured data refers to things that has a defined meaning such as price, age +- Unstructured data refers to thing like pixel, raw audio, text. + +#### Why is deep learning taking off + +Deep learning is taking off due to a large amount of data available through the digitization of the society, faster computation and innovation in the development of neural network algorithm. + +*Two things have to be considered to get to the high level of performance*: + +1. Being able to train a big enough neural network +2. Huge amount of labeled data + +## Week 2: Neural Networks Basics + +### Learning Objectives + +- Build a logistic regression model structured as a shallow neural network +- Build the general architecture of a learning algorithm, including parameter initialization, cost function and gradient calculation, and optimization implemetation (gradient descent) +- Implement computationally efficient and highly vectorized versions of models +- Compute derivatives for logistic regression, using a backpropagation mindset +- Use Numpy functions and Numpy matrix/vector operations +- Work with iPython Notebooks +- Implement vectorization across multiple training examples + +### Logistic Regression as a Neural Network + +#### Binary Classification + +Week2 focuses on the basics of neural network programming, especially some important techniques, such as how to deal with m training examples in the computation and how to implement forward and backward propagation. To illustrate this process step by step, Andrew Ng took a lot of time explaining how Logistic regression is implemented for a binary classification, here a Cat vs. Non-Cat classification, which would take an image as an input and output a label to propagation whether this image is a cat (label 1) or not (label 0). + +An image is store in the computer in three separate matrices corresponding to the Red, Green, and Blue color channels of the image. The three matrices have the same size as the image, for example, the resolution of the cat image is 64 pixels x 64 pixels, the three matrices (RGB) are 64 by 64 each. To create a feature vector, x, the pixel intensity values will be “unroll” or “reshape” for each color. The dimension of the input feature vector x is ![n_x=64\times64\times3=12288](../_resources/n_x_64_64_3_12288.svg). + +#### Logistic Regression + +> Logistic regression is useful for situations in which you want to be able to predict the presence or absence of a characteristic or outcome based on values of a set of predictor variables. It is similar to a linear regression model but is suited to models where the dependent variable is dichotomous. Logistic regression coefficients can be used to estimate odds ratios for each of the independent variables in the model. Logistic regression is applicable to a broader range of research situations than discriminant analysis. (from [ibm knowledge center](https://www.ibm.com/support/knowledgecenter/en/SSLVMB_26.0.0/statistics_mainhelp_ddita/spss/regression/idh_lreg.html)) + +A detailed guide on [Logistic Regression for Machine Learning](https://machinelearningmastery.com/logistic-regression-for-machine-learning/) by Jason Brownlee is the best summary of this topic for data science engineers. + +Andrew Ng's course on Logistic Regression here focuses more on LR as the simplest neural network, as its programming implementation is a good starting point for the deep neural networks that will be covered later. + +#### Logistic Regression Cost Function + +In Logistic regression, we want to train the parameters `w` and `b`, we need to define a cost function. + +![lr-eqn](../_resources/lr-eqn.svg), where ![lr-sigmoid](../_resources/lr-sigmoid.svg) +Given ![lr-input](../_resources/lr-input.svg), we want ![lr-target](../_resources/lr-target.svg) + +The loss function measures the discrepancy between the prediction (𝑦̂(𝑖)) and the desired output (𝑦(𝑖)). In other words, the loss function computes the error for a single training example. + +![lr-loss-function](../_resources/lr-loss-function.png) + +The cost function is the average of the loss function of the entire training set. We are going to find the parameters 𝑤 𝑎𝑛𝑑 𝑏 that minimize the overall cost function. + +![lr-cost-function](../_resources/lr-cost-function.png) + +The loss function measures how well the model is doing on the single training example, whereas the cost function measures how well the parameters w and b are doing on the entire training set. + +#### Gradient Descent + +As you go through any course on machine learning or deep learning, gradient descent the concept that comes up most often. It is used when training models, can be combined with every algorithm and is easy to understand and implement. + +The goal of the training model is to minimize the loss function, usually with randomly initialized parameters, and using a gradient descent method with the following main steps. Randomization of parameters initialization is not necessary in logistic regression (zero initialization is fine), but it is necessary in multilayer neural networks. + +1. Start calculating the cost and gradient for the given training set of (x,y) with the parameters w and b. +2. update parameters w and b with pre-set learning rate: + w\_new =w\_old – learning\_rate * gradient\_of\_at(w\_old) + Repeat these steps until you reach the minimal values of cost function. + +The fancy image below comes from [analytics vidhya](https://www.analyticsvidhya.com/blog/2020/10/what-does-gradient-descent-actually-mean/). + +![gradient-descent](../_resources/gradient-descent.jpeg) + +#### Derivatives + +Derivatives are crucial in backpropagation during neural network training, which uses the concept of computational graphs and the chain rule of derivatives to make the computation of thousands of parameters in neural networks more efficient. + +#### Computation Graph + +A nice illustration by [colah's blog](https://colah.github.io/posts/2015-08-Backprop/) can help better understand. + +Computational graphs are a nice way to think about mathematical expressions. For example, consider the expression e=(a+b)∗(b+1). There are three operations: two additions and one multiplication. To help us talk about this, let’s introduce two intermediary variables, c and d so that every function’s output has a variable. We now have: + +``` +c=a+b +d=b+1 +e=c∗d +``` + +To create a computational graph, we make each of these operations, along with the input variables, into nodes. When one node’s value is the input to another node, an arrow goes from one to another. + +![tree-def](../_resources/tree-def.png) + +#### Derivatives with a Computation Graph + +If one wants to understand derivatives in a computational graph, the key is to understand derivatives on the edges. If a directly affects c, then we want to know how it affects c. If a changes a little bit, how does c change? We call this the partial derivative of c with respect to a. + +![tree-eval-derivs](../_resources/tree-eval-derivs.png) + +#### Logistic Regression Gradient Descent + +Andrew did logistic regreesion gradient descent computation using the computation graph in order to get us familiar with computation graph ideas for neural networks. + +#### Gradient Descent on m Examples + +The cost function is computed as an average of the `m` individual loss values, the gradient with respect to each parameter should also be calculated as the mean of the `m` gradient values on each example. + +The calculattion process can be done in a loop through m examples. + +```python +J=0 +dw=np.zeros(n) +db=0 + +for i in range(m): + z[i] = w.transpose() * x[i] + b + a[i] = sigmoid(z[i]) + J = J + (-[y[i]*log(a[i])+(1-y[i])*log(1-a[i])]) + dz[i] = a[i] - y[i] + + # inner loop for n features, later will be optimized by vectorization + for j in range(n): + dw[j] = dw[j] + x[i][j] * dz[i] + + db = db + dz[i] + +j = j / m +dw = dw / m +db = db / m +``` + +After gradient computation, we can update parameters with a learning rate `alpha`. + +``` +# vectorization should also applied here +for j in range(n): + w[j] = w[j] - alpha * dw[j] +b = b - alpha * db +``` + +As you can see above, to update parameters one step, we have to go throught all the `m` examples. This will be mentioned again in later videos. Stay tuned! + +#### Derivation of dL/dz + +You may be wondering why `dz=a-y` in the above code is calculated this way and where it comes from. Here is a [detailed derivation process of dl/dz](https://www.coursera.org/learn/neural-networks-deep-learning/discussions/weeks/2/threads/ysF-gYfISSGBfoGHyLkhYg) on discussion forum. + +### Python and Vectorization + +#### Vectorization + +Both GPU and CPU have parallelization instructions. They're sometimes called SIMD instructions, which stands for a single instruction multiple data. The rule of thumb to remember is whenever possible avoid using explicit four loops. + +#### Vectorizing Logistic Regression + +If we stack all the `m` examples of `x` we have a input matrix `X` with each column representing an example. So by the builtin vectorization of numpy we can simplify the above gradient descent calculation with a few lines of code which can boost the computational efficiency definitely. + +``` +Z = np.dot(w.T, X) + b +A = sigmoid(Z) +dz = A - Y + +# in constrast to the inner loop above, vectorization is used here to boost computation +dw = 1/m * np.dot(X, dz.T) +db = 1/m * np.sum(dz) +``` + +Update parameters: + +``` +w = w - alpha * dw +b = b - alpha * db +``` + +#### Broadcasting in Python + +The term broadcasting describes how numpy treats arrays with different shapes during arithmetic operations. Subject to certain constraints, the smaller array is "broadcast" across the larger array so that they have compatible shapes. Broadcasting provides a means of vectorizing array operations so that looping occurs in C instead of Python. More detailed examples on [numpy.org](https://numpy.org/doc/stable/user/theory.broadcasting.html#array-broadcasting-in-numpy). + +![theory.broadcast_2.gif](../_resources/theory.broadcast_2.gif) + +#### A note on python/numpy vectors + +Do not use rank 1 arrays: + +``` +# an example of rank 1 array +a = np.random.randn(5) +a.shape +# (5,) +``` + +Instead, we should use these: + +``` +a = np.random.randn(5,1) +a = np.random.randn(1,5) +``` + +Or, just reshape the first case if necessary: + +``` +a = a.reshape(5,1) +a.shape +# (5,1) +``` + +#### Quick tour of Jupyter/iPython Notebooks + +The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more. + +JupyterLab is a web-based interactive development environment for Jupyter notebooks, code, and data. JupyterLab is flexible: configure and arrange the user interface to support a wide range of workflows in data science, scientific computing, and machine learning. JupyterLab is extensible and modular: write plugins that add new components and integrate with existing ones. + +See [jupyter.org](https://jupyter.org/) + +``` +pip install jupyterlab +``` + +To open jupyter lab, run + +``` +jupyter-lab +``` + +#### Explanation of logistic regression cost function (optional) + +But so to summarize, by minimizing this cost function J(w,b) we're really carrying out maximum likelihood estimation with the logistic regression model because minimizing the loss corresponds to maximizing the log of the probability. + +![prob](../_resources/prob-conditional.svg) + +![prob-cost](../_resources/prob-cost.svg) + +## Week 3: Shallow Neural Networks + +### Learning Objectives + +- Describe hidden units and hidden layers +- Use units with a non-linear activation function, such as tanh +- Implement forward and backward propagation +- Apply random initialization to your neural network +- Increase fluency in Deep Learning notations and Neural Network Representations +- Implement a 2-class classification neural network with a single hidden layer + +### Shallow Neural Network + +#### Neural Networks Overview + +This is a simple 2-layer neural network (just one hidden layer) + +![neural-network-2-layer](../_resources/neural-network-2-layer.png) + +Using computation graph, the forward computation process is like this. + +![neural-network-2-layer-forward](../_resources/neural-network-2-layer-forward.png) + +#### Neural Network Representation + +A neural network consists of three types of layers: input layer, hidden layer and output layer. Input layer is not counted in the number of layers of one neural network. When we talk about training a neural network, basically we are training parameters associated with the hidden layers and the output layer. + +- Input layer: input features (x1, x2, x3, ...) stack up vertically +- Hidden layer(s): values for the nodes are not observed +- Output layer: responsilble for generating the predicted value + +![nn-representation](../_resources/nn-representation.png) + +#### Computing a Neural Network's Output + +![nn-computation](../_resources/nn-computation.png) + +In the above example, `z[1]` is the result of linear computation of the input values and the parameters of the hidden layer and `a[1]` is the activation as a sigmoid function of `z[1]`. + +Generally, in a two-layer neural network, if we have `nx` features of input `x` and `n1` neurons of hidden layer and one output value, we have the following dimensions of each variable. Specifically, we have `nx=3, n1=4` in the above network. + +| variable | shape | description | +| --- | --- | --- | +| `x` | `(nx,1)` | input value with `nx` features | +| `W[1]` | `(n1,nx)` | weight matrix of first layer, i.e., hidden layer | +| `b[1]` | `(n1,1)` | bias terms of hidden layer | +| `z[1]` | `(n1,1)` | result of linear computation of hidden layer | +| `a[1]` | `(n1,1)` | activation of hidden layer | +| `W[2]` | `(1,n1)` | weight matrix of second layer, i.e., output layer here | +| `b[2]` | `(1,1)` | bias terms of output layer | +| `z[2]` | `(1,1)` | result of linear computation of output layer | +| `a[2]` | `(1,1)` | activation of output layer, i.e., output value | + +We should compute `z[1], a[1], z[2], a[2]` for each example `i` of `m` examples: + +``` +for i in range(m): + z[1][i] = W[1]*x[i] + b[1] + a[1][i] = sigmoid(z[1][i]) + z[2][i] = W[2]*a[1][i] + b[2] + a[2][i] = sigmoid(z[2][i]) +``` + +#### Vectorizing across multiple examples + +Just as we have already been familiar with vectorization and broadcasting in the logistic regression, we can also apply the same method to the neural networks training. Inevitably, we have to go through the `m` examples of input values in the process of computation. Stacking them together is good idea. So we have the following vectorizing variables with only small differences as before. + +| variable | shape | description | +| --- | --- | --- | +| `X` | `(nx,m)` | input values with `nx` features | +| `W[1]` | `(n1,nx)` | weight matrix of first layer, i.e., hidden layer | +| `b[1]` | `(n1,1)` | bias terms of hidden layer | +| `Z[1]` | `(n1,m)` | results of linear computation of hidden layer | +| `A[1]` | `(n1,m)` | activations of hidden layer | +| `W[2]` | `(1,n1)` | weight matrix of second layer, i.e., output layer here | +| `b[2]` | `(1,1)` | bias terms of output layer | +| `Z[2]` | `(1,1)` | results of linear computation of output layer | +| `A[2]` | `(1,1)` | activations of output layer, i.e., output value | + +Now we can compute `Z[1], A[1], Z[2], A[2]` all at once. + +``` +Z[1] = W[1]*X + b[1] +A[1] = sigmoid(Z[1]) +Z[2] = W[2]*A[1] + b[2] +A[2] = sigmoid(Z[2]) +``` + +#### Activation functions + +So far, we know that a non-linear function is applied in the output step of each layer. Actually there are several common activation functions which are also popular. + +| activation | formula | graph | description | +| --- | --- | --- | --- | +| sigmoid | ![a=1/(1+np.exp(-z))](../_resources/sigmoid-latex.svg) | ![sigmoid.png](../_resources/sigmoid.png) | also called logistic activation function, looks like an S-shape, if your output value between 0 and 1 choose sigmoid | +| tanh | ![a=(np.exp(z)-np.exp(-z))/(np.exp(z)+np.exp(-z))](../_resources/tanh-latex.svg) | ![tanh.png](../_resources/tanh.png) | tanh activation usually works better than sigmoid activation function for hidden units because the mean of its output is closer to zero, and so it centers the data better for the next layer | +| ReLU | `a=max(0,z)` | ![tanh.png](../_resources/relu.png) | rectified linear unit, the most widely used activation function | +| Leaky ReLU | `a=max(0.01z,z)` | ![tanh.png](../_resources/leaky-relu.png) | an improved version of ReLU, 0.01 can be a parameter | + +#### Why do you need non-linear activation functions + +> If we only allow linear activation functions in a neural network, the output will just be a linear transformation of the input, which is not enough to form a universal function approximator. Such a network can just be represented as a matrix multiplication, and you would not be able to obtain very interesting behaviors from such a network. + +A good explanation on [Stack Overflow](https://stackoverflow.com/questions/9782071/why-must-a-nonlinear-activation-function-be-used-in-a-backpropagation-neural-net). + +#### Derivatives of activation functions + +| activation | formula | derivative | +| --- | --- | --- | +| sigmoid | ![a=1/(1+np.exp(-z))](../_resources/sigmoid-latex.svg) | a(1-a) | +| tanh | ![a=(np.exp(z)-np.exp(-z))/(np.exp(z)+np.exp(-z))](../_resources/tanh-latex.svg) | 1-a^2 | +| ReLU | `a=max(0,z)` | 0 if z<0; 1 if z>=0 | +| Leaky ReLU | `a=max(0.01z,z)` | 0.01 if z<0; 1 if z>=0 | + +#### Gradient descent for Neural Networks + +Again we will have a single hidden layer in our neural network, this section focuses on the equations we need to implement in order to get back-propagation or to get gradient descent working. Suppose we have nx input features, n1 hidden units and n2 output units in our examples. In the previous vectorization section we have n2 equals one. Here we will have a more general representation in order to give ourselves a smoother transition to the next week of the course. + +*Variables*: + +| variable | shape | description | +| --- | --- | --- | +| `X` | `(nx,m)` | input values with `nx` features | +| `Z[1]` | `(n1,m)` | results of linear computation of hidden layer | +| `A[1]` | `(n1,m)` | activations of hidden layer | +| `Z[2]` | `(n2,1)` | results of linear computation of output layer | +| `A[2]` | `(n2,1)` | activations of output layer, i.e., output value | + +*Parameters*: + +| variable | shape | description | +| --- | --- | --- | +| `W[1]` | `(n1,nx)` | weight matrix of first layer, i.e., hidden layer | +| `b[1]` | `(n1,1)` | bias terms of hidden layer | +| `W[2]` | `(n2,n1)` | weight matrix of second layer, i.e., output layer here | +| `b[2]` | `(n2,1)` | bias terms of output layer | + +*Forward propagation* computes all the variable values of each layer, which will also be used in the backpropagation computation. + +``` +Z[1] = W[1]*X + b[1] +A[1] = sigmoid(Z[1]) +Z[2] = W[2]*A[1] + b[2] +A[2] = sigmoid(Z[2]) +``` + +*Backpropagation* computes the derivatives of parameters by the chain rule. + +``` +# backpropagation +dZ[2] = A[2] - Y # get this with combination of the derivative of cost function and g'[2] +dW[2] = 1/m * np.matmul(dZ[2], A[1].T) +db[2] = 1/m * np.sum(dZ[2], axis=1, keepdims=True) +dZ[1] = np.multiply(np.matmul(W[2].T, dZ[2]), g'[1](Z[1])) # derivative of activation is used here +dW[1] = 1/m * np.matmul(dZ[1], X.T) +db[1] = 1/m * np.sum(dZ[1]) + +# update parameters +W[1] = W[1] - learning_rate * dW[1] +b[1] = b[1] - learning_rate * db[1] +W[2] = W[2] - learning_rate * dW[2] +b[2] = b[2] - learning_rate * db[2] +``` + +*Repeat* forward propagation and backpropagation a lot of times until the parameters look like they're converging. + +#### Random initialization + +Initialization of parameters: + +``` +W[1] = np.random.randn((n1,nx)) * 0.01 # randomized small numbers +b[1] = np.zeros((n1,1)) # zeros is fine for bias terms +W[2] = np.random.randn((n2,n1)) * 0.01 +b[2] = np.zeros((n2,1)) +``` + +*Why randomized initialization?* + +In order to break the symmetry for hidden layers. + +> Imagine that you initialize all weights to the same value (e.g. zero or one). In this case, each hidden unit will get exactly the same signal. E.g. if all weights are initialized to 1, each unit gets signal equal to sum of inputs (and outputs sigmoid(sum(inputs))). If all weights are zeros, which is even worse, every hidden unit will get zero signal. No matter what was the input - if all weights are the same, all units in hidden layer will be the same too. + +See some interesting discussion on [Stack Overflow](https://stackoverflow.com/questions/20027598/why-should-weights-of-neural-networks-be-initialized-to-random-numbers). + +*Why small numbers?* + +This is for sigmoid or tanh activation function. If weight parameters are initially large, we are more likely to get large values of `z` calculated by `z=wx+b`. If we check this in the graph of sigmoid(tanh) function, we can see the slope in large `z` is very close to zero, which would slow down the learning process since parameters are updated by only a very small amount each time. + +## Week 4: Deep Neural Networks + +### Learning Objectives + +- Describe the successive block structure of a deep neural network +- Build a deep L-layer neural network +- Analyze matrix and vector dimensions to check neural network implementations +- Use a cache to pass information from forward to back propagation +- Explain the role of hyper-parameters in deep learning + +### Deep Neural Network + +#### Deep L-layer neural network + +Technically logistic regression is a 1-layer neural network. Deep neural networks, with more layers, can learn functions that shallower models are often unable to. + +Here `L` denotes the number of layers in a deep neural network. Some notations: + +| notation | description | +| --- | --- | +| `n[0]` | number of neurons in the input layer | +| `n[l]` | number of neurons in the `lth` layer, `l` from 1 to L | +| `W[l]` | weights of the l-layer of shape `(n[l], n[l-1])` | +| `b[l]` | bias term of the l-layer of shape `(n[l], 1)` | +| `Z[l]` | affine result of the l-layer of shape `(n[l], m)`, `Z[l]=W[l]A[l-1]+b[l]` | +| `g[l]` | activation function of the l-layer | +| `A[l]` | activation output of the l-layer of shape `(n[l], m)`, `A[l]=g[l](Z[l])` | + +#### Forward Propagation in a deep network + +With `A[0]=X`, forward propagation is generalized as: + +``` +Z[l] = W[l]*A[l-1] + b[l] +A[l] = sigmoid(Z[l]) +``` + +*Backpropagation* computes the derivatives of parameters by the chain rule. + +``` +# backpropagation +dZ[2] = A[2] - Y # get this with combination of the derivative of cost function and g'[2] +dW[2] = 1/m * np.matmul(dZ[2], A[1].T) +db[2] = 1/m * np.sum(dZ[2], axis=1, keepdims=True) +dZ[1] = np.multiply(np.matmul(W[2].T, dZ[2]), g'[1](Z[1])) # derivative of activation is used here +dW[1] = 1/m * np.matmul(dZ[1], X.T) +db[1] = 1/m * np.sum(dZ[1]) + +# update parameters +W[1] = W[1] - learning_rate * dW[1] +b[1] = b[1] - learning_rate * db[1] +W[2] = W[2] - learning_rate * dW[2] +b[2] = b[2] - learning_rate * db[2] +``` + +#### Getting your matrix dimensions right + +| matrix | dimension | +| --- | --- | +| `W[l]` | `(n[l], n[l-1])` | +| `b[l]` | `(n[l], 1)` | +| `Z[l]` | `(n[l], m)` | +| `A[l]` | `(n[l], m)` | +| `dW[l]` | `(n[l], n[l-1])` | +| `db[l]` | `(n[l], 1)` | +| `dZ[l]` | `(n[l], m)` | +| `dA[l]` | `(n[l], m)` | + +#### Why deep representations + +- Deep neural network with multiple hidden layers might be able to have the earlier layers learn lower level simple features and then have the later deeper layers then put together the simpler things it's detected in order to detect more complex things like recognize specific words or even phrases or sentences. +- If there aren't enough hidden layers, then we might require exponentially more hidden units to compute in shallower networks. + +#### Building blocks of deep neural networks + +![nn framework](../_resources/nn_frame.png) + +*Implementation steps*: + +1. Initialize parameters / Define hyperparameters +2. Loop for num_iterations: + 1. Forward propagation + 2. Compute cost function + 3. Backward propagation + 4. Update parameters (using parameters, and grads from backprop) +3. Use trained parameters to predict labels + +#### Forward and Backward Propagation + +In the algorithm implementation, outputting intermediate values as caches (basically `Z` and `A`) of each forward step is crucial for backward computation. + +![forward and backward](../_resources/backprop_flow.png) + +#### Parameters vs Hyperparameters + +*Parameters*: + +- weight matrices `W` of each layer +- bias terms `b` of each layer + +*Hyper parameters*: + +- number of hidden units `n[l]` +- learning rate +- number of iteration +- number of layers `L` +- choice of activation functions + +### What does this have to do with the brain + +About this topic, I think the following Andrew's explanation is the best summary: + +> I do think that maybe the few that computer vision has taken a bit more inspiration from the human brain than other disciplines that also apply deep learning, but I personally use the analogy to the human brain less than I used to. \ No newline at end of file diff --git a/DeepLearning_AI/Course 2_ Improving Deep Neural Networks_ Hyperpar.md b/DeepLearning_AI/Course 2_ Improving Deep Neural Networks_ Hyperpar.md new file mode 100644 index 0000000..316b417 --- /dev/null +++ b/DeepLearning_AI/Course 2_ Improving Deep Neural Networks_ Hyperpar.md @@ -0,0 +1,677 @@ +--- +title: >- + Course 2: Improving Deep Neural Networks: Hyperparameter tuning, + Regularization and Optimization +updated: 2022-05-23 11:00:22Z +created: 2022-05-16 17:54:09Z +--- + +# Course 2: Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization + +- [Course 2: Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization](#course-2-improving-deep-neural-networks-hyperparameter-tuning-regularization-and-optimization) + - [Week 1: Practical aspects of Deep Learning](#week-1-practical-aspects-of-deep-learning) + - [Learning Objectives](#learning-objectives) + - [Setting up your Machine Learning Application](#setting-up-your-machine-learning-application) + - [Train / Dev / Test sets](#train--dev--test-sets) + - [Bias / Variance](#bias--variance) + - [Basic Recipe for Machine Learning](#basic-recipe-for-machine-learning) + - [Regularizing your neural network](#regularizing-your-neural-network) + - [Regularization](#regularization) + - [Why regularization reduces over-fitting](#why-regularization-reduces-over-fitting) + - [Dropout Regularization](#dropout-regularization) + - [Understanding Dropout ("Inverted Dropout")](#understanding-dropout-inverted-dropout) + - [Other regularization methods](#other-regularization-methods) + - [Setting up your optimization problem](#setting-up-your-optimization-problem) + - [Normalizing inputs](#normalizing-inputs) + - [Vanishing / Exploding gradients](#vanishing--exploding-gradients) + - [Weight Initialization for Deep Networks](#weight-initialization-for-deep-networks) + - [Numerical approximation of gradients](#numerical-approximation-of-gradients) + - [Gradient checking](#gradient-checking) + - [Gradient checking implementation notes](#gradient-checking-implementation-notes) + - [Week 2: Optimization algorithms](#week-2-optimization-algorithms) + - [Learning Objectives](#learning-objectives-1) + - [Optimization algorithms](#optimization-algorithms) + - [Mini-batch gradient descent](#mini-batch-gradient-descent) + - [Understanding mini-batch gradient descent](#understanding-mini-batch-gradient-descent) + - [Exponentially Weighted Averages](#exponentially-weighted-averages) + - [Understanding exponentially weighted averages](#understanding-exponentially-weighted-averages) + - [Bias correction in exponentially weighted averages](#bias-correction-in-exponentially-weighted-averages) + - [Gradient descent with momentum](#gradient-descent-with-momentum) + - [RMSprop](#rmsprop) + - [Adam optimization algorithm](#adam-optimization-algorithm) + - [Learning rate decay](#learning-rate-decay) + - [The problem of local optima](#the-problem-of-local-optima) + - [Quick notes for optimization algorithms](#quick-notes-for-optimization-algorithms) + - [Week 3: Hyperparameter tuning, Batch Normalization and Programming Frameworks](#week-3-hyperparameter-tuning-batch-normalization-and-programming-frameworks) + - [Learning Objectives](#learning-objectives-2) + - [Hyperparameter tuning](#hyperparameter-tuning) + - [Tuning process](#tuning-process) + - [Using an appropriate scale to pick hyperparameters](#using-an-appropriate-scale-to-pick-hyperparameters) + - [Hyperparameters tuning in practice: Panda vs. Caviar](#hyperparameters-tuning-in-practice-panda-vs-caviar) + - [Batch Normalization](#batch-normalization) + - [Normalizing activations in a network](#normalizing-activations-in-a-network) + - [Fitting Batch Norm into a neural network](#fitting-batch-norm-into-a-neural-network) + - [Why does Batch Norm work](#why-does-batch-norm-work) + - [Batch Norm at test time](#batch-norm-at-test-time) + - [Multi-class classification](#multi-class-classification) + - [Softmax Regression](#softmax-regression) + - [Training a softmax classifier](#training-a-softmax-classifier) + - [Introduction to programming frameworks](#introduction-to-programming-frameworks) + - [Deep learning frameworks](#deep-learning-frameworks) + - [Tensorflow](#tensorflow) + +## Week 1: Practical aspects of Deep Learning + +### Learning Objectives + +- Give examples of how different types of initializations can lead to different results +- Examine the importance of initialization in complex neural networks +- Explain the difference between train/dev/test sets +- Diagnose the bias and variance issues in your model +- Assess the right time and place for using regularization methods such as dropout or L2 regularization +- Explain Vanishing and Exploding gradients and how to deal with them +- Use gradient checking to verify the accuracy of your back-propagation implementation + +### Setting up your Machine Learning Application + +#### Train / Dev / Test sets + +Setting up the training, development (dev, also called validate set) and test sets has a huge impact on productivity. It is important to choose the dev and test sets from the same distribution and it must be taken randomly from all the data. +![b190151ee52f3e4c4e48f19afe65db7a.png](../_resources/b190151ee52f3e4c4e48f19afe65db7a.png) +In Big Data (> 1.000.000 m) take +|Training|dev|test| +|-|-|-| +|98%|1%|1%| +|99,5%|0,5%|0,5%| + +*Guideline*: + +- Choose a dev set and test set to reflect data you expect to get in the future. +- The dev and test sets should be just big enough to represent accurately the performance of the model. +- __Make sure dev and test set come from the same distribution__ +- Test set is not always necessary + +#### Bias / Variance + +![5563a5e13ec8cdd9e44831e7209880a2.png](../_resources/5563a5e13ec8cdd9e44831e7209880a2.png) + +| error type | high variance | high bias | high bias, high variance | low bias, low variance | +| --- | --- | --- | --- | --- | +| Train set error | 1% | 15% | 15% | 0.5% | +| Dev set error | 11% | 16% | 30% | 1% | + +> When we discuss prediction models, prediction errors can be decomposed into two main subcomponents we care about: error due to "bias" and error due to "variance". There is a tradeoff between a model's ability to minimize bias and variance. Understanding these two types of error can help us diagnose model results and avoid the mistake of over- or under-fitting. + +If human error is $\approx$ 0% then optimal (Bayes) error $\approx$ 0. So percentages in table above are relative. When human error is 15% all looks different. + +![46f3e8c2b3038d9c4b19536c9cdd9a21.png](../_resources/46f3e8c2b3038d9c4b19536c9cdd9a21.png) +High bias because is not fitting the green line area. + +To understand bias and variance better, read this essay: [Understanding the Bias-Variance Tradeoff](http://scott.fortmann-roe.com/docs/BiasVariance.html). + +#### Basic Recipe for Machine Learning + +![bias-variance-tradeoff](../_resources/bias-variance-tradeoff.png) + +- For a high bias problem, getting more training data is actually not going to help. +- Back in the pre-deep learning era, we didn't have as many tools that just reduce bias or that just reduce variance without hurting the other one. +- In the modern deep learning, big data era, getting a bigger network and more data almost always just __reduces bias without necessarily hurting your variance__, so long as you regularize appropriately. +- This has been one of the big reasons that deep learning has been so useful for supervised learning. +- The main cost of training a big neural network is just computational time, so long as you're regularizing. +- Useful reducing variance: + - L2 regularization + - Data augmentation + - Dropout + - More data + +### Regularizing your neural network + +#### Regularization + +__Regularization for Logistic Regression__: + +![reg-cost](../_resources/reg-logistic-cost.svg) +__strong text__ +`b` is just one parameter over a very large number of parameters, so no need to include it in the regularization. + +| regularization | formula | description | +| --- | --- | --- | +| L2 regularization | ![reg-cost](../_resources/reg-logistic-l2.svg) | most common type of regularization | +| L1 regularization | ![reg-cost](../_resources/reg-logistic-l1.svg) | w vector will have a lot of zeros, so L1 regularization makes your model sparse | + +__Regularization for a Neural Network__: + +![reg-cost](../_resources/reg-nn-cost.svg) + +For the matrix `w`, this norm is called the Frobenius norm. Its definition looks like `L2` norm but is not called the `L2` norm: + +![reg-cost](../_resources/reg-nn-fnorm.svg) + +Regularization of gradient: + +![reg-nn-grad](../_resources/reg-nn-grad.svg) + +With regularization the coefficient of `w` is slightly less than `1`, in which case it is called __weight decay__. + +![reg-nn-weight-decay](../_resources/reg-nn-wdecay.svg) + +#### Why regularization reduces over-fitting + +- If we make regularization lambda to be very big, then weight matrices will be set to be reasonably close to zero, __effectively zeroing out a lot of the impact of the hidden units.__ Then the simplified neural network becomes a much smaller neural network, eventually almost like a logistic regression. We'll end up with a much smaller network that is therefore less prone to over-fitting. +- Taking activation function `g(Z)=tanh(Z)` as example, if lambda is large, then weights `W` are small and subsequently `Z` ends up taking relatively small values, where `g` and `Z` will be roughly linear which is not able to fit those very complicated decision boundary, i.e., less able to over-fit. +![c2e4a299ff489309dab8e6e81864d82b.png](../_resources/c2e4a299ff489309dab8e6e81864d82b.png) + +*Implementation tips*: + +Without regularization term, we should see the cost function decreases monotonically in the plot. Whereas in the case of regularization, to debug gradient descent make sure that we plot `J` with a regularization term; otherwise, if we plot only the first term (the old J), we might not see a decrease monotonically. +![35ae293edf8fad30686a64bbe5dcc90a.png](../_resources/35ae293edf8fad30686a64bbe5dcc90a.png) + +#### Dropout Regularization + +- Dropout is another powerful regularization technique. +- With dropout, what we're going to do is go through each of the layers of the network and set some probability of eliminating a node in neural network. It's as if on every iteration you're working with a smaller neural network, which has a regularizing effect. +- Inverted dropout technique, `a3 = a3 / keep_prob`, ensures that the expected value of `a3` remains the same, which makes test time easier because you have less of a scaling problem. + +![dropout](../_resources/dropout.jpeg) + +*(image source: [deepnotes](https://deepnotes.io/dropout))* + +#### Understanding Dropout ("Inverted Dropout") + +- Can't rely on any one feature, so have to spread out weights, which has an effect of shrinking the squared norm of the weights, similar to what we saw with L2 regularization, helping prevent over-fitting. +- For layers where you're more worried about over-fitting, really the layers with a lot of parameters, you can set the key prop to be smaller to apply a more powerful form of drop out. +- Downside: with `keep prop` for some layers, more hyper-parameters to search for using cross-validation. +- Frequently used in __computer vision__, as the input size is so big, inputting all these pixels that you almost never have enough data, prone to over-fitting. +- Cost function `J` is no longer well-defined and harder to debug or double check that `J` is going downhill on every iteration. So first run code and make sure old `J` is monotonically decreasing, and then turn on drop out in order to make sure that no bug in drop out. +- Do not use at test runs +- makes tests easier because less scaling problem. +- different runs, different random zero out! + +*Note*: + +- A __common mistake__ when using dropout is to use it both in training and testing. You should use dropout (randomly eliminate nodes) only in training. +- Turn off dropout during testing (keep.prob = 1.0) +- Deep learning frameworks like [tensorflow](https://www.tensorflow.org/api_docs/python/tf/nn/dropout), [PaddlePaddle](https://www.paddlepaddle.org.cn/documentation/docs/en/api/layers/dropout.html), [keras](https://keras.io/api/layers/regularization_layers/dropout/) or [caffe](http://caffe.berkeleyvision.org/tutorial/layers/dropout.html) come with a dropout layer implementation. Don't stress - you will soon learn some of these frameworks. + +#### Other regularization methods + +- __Data augmentation__: getting more training data can be expensive and sometimes can't get more data, so flipping horizontally, random cropping, random distortion and translation of image can make additional fake training examples. +- __Early stopping__: stopping halfway to get a mid-size `w`. + - *Disadvantage*: early stopping couples two tasks of machine learning, optimizing the cost function `J` and not over-fitting, which are supposed to be completely separate tasks, to make things more complicated. + - __orthogonalization__: think on ONE task at the time. + - *Advantage*: running the gradient descent process just once, you get to try out values of small `w`, mid-size `w`, and large `w`, without needing to try a lot of values of the L2 regularization hyper-parameter lambda. + - focus on + - first: Optimize cost function J + - then: not overfit (L2 etc) +![f96c721f766385e5fc346faf159dcd20.png](../_resources/f96c721f766385e5fc346faf159dcd20.png) + +### Setting up your optimization problem + +#### Normalizing inputs + +With normalization, cost function will be more round and easier to optimize when features are all on similar scales. This is a very common topic, see more on [Stack Overflow](https://stackoverflow.com/questions/4674623/why-do-we-have-to-normalize-the-input-for-an-artificial-neural-network). + +- faster learning rate gradient decent +- use same normalization for train and test +![a157a98c23f8955d06c56f8be279c074.png](../_resources/a157a98c23f8955d06c56f8be279c074.png) + +#### Vanishing / Exploding gradients + +- In a very deep network derivatives or slopes can sometimes get either very big or very small, maybe even exponentially, and this makes training difficult. +- The weights W, if they're all just a little bit bigger than one or just a little bit bigger than the identity matrix, then with a very deep network the activations can explode. And if W is just a little bit less than identity, the activations will decrease exponentially. + +#### Weight Initialization for Deep Networks + +A partial solution to the problems of vanishing and exploding gradients is better or more careful choice of the random initialization for neural network. + +For a single neuron, suppose we have `n` features for the input layer, then we want `Z = W1X1 + W2X2 + ... + WnXn` not blow up and not become too small, so the larger `n` is, the smaller we want `Wi` to be. + +- It's reasonable to set variance of `Wi` to be equal to `1/n` +- It helps reduce the vanishing and exploding gradients problem, because it's trying to set each of the weight matrices `W` not too much bigger than `1` and not too much less than `1`. +- Generally for layer `l`, set `W[l]=np.random.randn(shape) * np.sqrt(1/n[l-1])`. + - For `relu` activation, set `Var(W)=2/n` by `W[l]=np.random.randn(shape) * np.sqrt(2/n[l-1])`. (aka He initialization by [Kaiming He](http://kaiminghe.com/)) + - For `tanh` activation, `W[l]=np.random.randn(shape) * np.sqrt(1/n[l-1])`. (Xavier initialization) + - `W[l]=np.random.randn(shape) * np.sqrt(2/(n[l-1]+n[l]))` (Yoshua Bengio) +- `1` or `2` in variance `Var(W)=1/n or 2/n` can be a hyperparameter, but not as important as other hyperparameters. + +*A well chosen initialization can*: + +- Speed up the convergence of gradient descent +- Increase the odds of gradient descent converging to a lower training (and generalization) error + +*Implementation tips*: + +- The weights `W[l]` should be initialized randomly to *break symmetry* and make sure different hidden units can learn different things. Initializing all the weights to zero results in the network failing to break symmetry. This means that every neuron in each layer will learn the same thing. +- It is however okay to initialize the biases `b[l]` to zeros. Symmetry is still broken so long as `W[l]` is initialized randomly. +- Initializing weights to very large random values does not work well. +- Hopefully initializing with small random values does better. The important question is: how small should be these random values be? He initialization works well for networks with ReLU activations. In other cases, try other initializations. + +#### Numerical approximation of gradients + +Numerically verify implementation of derivative of a function is correct and hence to check if there is a bug in the back-propagation implementation. + +*Two-sided difference formula is much more accurate*: + +- In two side case, `f'(𝜃)=lim(f(𝜃+𝜀)-f(𝜃-𝜀))/(2𝜀), error term ~ O(𝜀^2)` +- In one side case, `f'(𝜃)=lim(f(𝜃+𝜀)-f(𝜃))/(𝜀), error term ~ O(𝜀)` +- `𝜀 < 1`, so `O(𝜀^2) < O(𝜀)` + +#### Gradient checking + +*Implementation steps*: + +1. Take `W[1],b[1],...,W[L],b[L]` and reshape into a big vector `𝜃`: `J(W[1],b[1],...,W[L],b[L])=J(𝜃)`. +2. Take `dW[1],db[1],...,dW[L],db[L]` and reshape into a big vector `d𝜃`. +3. For each `i`: `d𝜃_approx[i] = (J(𝜃1,𝜃2,...,𝜃i+𝜀,...)-J(𝜃1,𝜃2,...,𝜃i-𝜀,...))/(2𝜀)`. (Should have `d𝜃_approx[i] ≈ d𝜃[i]`) +4. Check `diff_ratio = norm_2(d𝜃_approx-d𝜃) / (norm_2(d𝜃_approx)+norm_2(d𝜃)) ≈ eps`: + 1. `diff_ratio ≈ 10^-7`, great, backprop is very likely correct. + 2. `diff_ratio ≈ 10^-5`, maybe OK, better check no component of this difference is particularly large. + 3. `diff_ratio ≈ 10^-3`, worry, check if there is a bug. + +#### Gradient checking implementation notes + +- Don't use in training - only to debug +- If algorithm fails grad check, look at components to try to identify bug. +- Remember regularization. +- Doesn't work with dropout. (you can first check grad, then turn on dropout) +- Run at random initialization; perhaps again after some training. + +## Week 2: Optimization algorithms + +### Learning Objectives + +- Apply optimization methods such as (Stochastic) Gradient Descent, Momentum, RMSProp and Adam +- Use random minibatches to accelerate convergence and improve optimization +- Describe the benefits of learning rate decay and apply it to your optimization + +### Optimization algorithms + +#### Mini-batch gradient descent + +Vectorization allows you to process all M examples relatively quickly if M is very large, but it can still be slow. For example, `m = 5,000,000` (or `m = 50,000,000` or even bigger), we have to process the entire training sets of five million training samples before we take one little step of gradient descent. + +We can use the mini-batch method to let gradient descent start to make some progress before we finish processing the entire, giant training set of 5 million examples by splitting up the training set into smaller, little baby training sets called mini-batches. In this case, we have 5000 mini-batches with 1000 examples each. + +*Notations*: + +- `(i)`: the *i*-th training sample +- `[l]`: the *l*-th layer of the neural network +- `{t}`: the *t*-th mini batch + +In every step of the iteration loop, we need to loop for `num_batches` and do forward and backward computation for each batch. + +1. Forward propagation +2. Compute cost function +3. Backward propagation +4. Update parameters (using parameters, and grads from backprop) + +With mini-batch gradient descent, a single pass through the training set is one epoch, which in the above 5 million example, means 5000 gradient descent steps. + +#### Understanding mini-batch gradient descent + +| batch size | method | description | guidelines | +| --- | --- | --- | --- | +| =`m` | batch gradient descent | cost function decreases on every iteration;
but too long per iteration. | for a small training set (<2000). | +| =`1` | stochastic gradient descent | cost function oscillates, can be extremely noisy;
wander around minimum;
lose speedup from vectorization, inefficient. | use a smaller learning rate when it oscillates too much. | +| between `1` and `m` | mini-batch gradient descent | somewhere in between, vectorization advantage, faster;
not guaranteed to always head toward the minimum but more consistently in that direction than stochastic descent;
not always exactly converge, may oscillate in a very small region, reducing the learning rate slowly may also help. | mini-batch size is a hyperparameter;
batch size better in \[64, 128, 256, 512\], a power of 2;
make sure that mini-batch fits in CPU/GPU memory. | + +#### Exponentially Weighted Averages + +Moving averages are favored statistical tools of active traders to measure momentum. There are three MA methods: + +| MA methods | calculations | +| --- | --- | +| simple moving average (SMA) | calculated from the average closing prices for a specified period | +| weighted moving average (WMA) | calculated by multiplying the given price by its associated weighting (assign a heavier weighting to more current data points) and totaling the values | +| exponential moving average (EWMA) | also weighted toward the most recent prices, but the rate of decrease is exponential | + +For a list of daily temperatures: + +![london-temp-example](../_resources/ewa-temp1.svg) + +This data looks a little bit noisy (blue dots): + +![ewa1](../_resources/ewa-temp-plot1.png) + +![ewa-on-temp](../_resources/ewa-temp2.svg) + +If we want to compute the trends, by averaging over a larger window, the above exponentially weighted average formula adapts more slowly when the temperature changes. So, there's just a bit more latency. (See the red curve above) + +- When `β=0.98` then it's giving a lot of weight to the previous value and a much smaller weight just 0.02, to whatever you're seeing right now. (See the green curve below) +- When `β=0.5`, which something like averaging over just two days temperature. And by averaging only over two days temperature, as if averaging over much shorter window. It's much more noisy, much more susceptible to outliers. But this adapts much more quickly to what the temperature changes. (See the yellow curve below) + +![ewa2](../_resources/ewa-temp-plot2.png) + +#### Understanding exponentially weighted averages + +This topic is basically related to [gradient descent optimizations](http://people.duke.edu/~ccc14/sta-663-2018/notebooks/S09G_Gradient_Descent_Optimization.html). + +![ewa](../_resources/ewa.svg) + +The exponentially weighted average adds a fraction β of the current value to a leaky running sum of past values. Effectively, the contribution from the *t−n*th value is scaled by ![ewa-weight](../_resources/ewa-weight.svg). + +For example, here are the contributions to the current value after 5 iterations (iteration 5 is the current iteration) + +| iteration | contribution | +| --- | --- | +| 1 | `β^4(1−β)` | +| 2 | `β^3(1−β)` | +| 3 | `β^2(1−β)` | +| 4 | `β^1(1−β)` | +| 5 | `(1−β)` | + +Since `β<1`, the contribution decreases exponentially with the passage of time. Effectively, this acts as a smoother for a function. + +***e*-folding**: + +Andrew Ng also mentioned an interesting concept related to *e*-folding. He said: + +- if `β=0.9` it would take about 10 days for `V` to decay to about `1/3` (`1/e ≈ 1/3`) of the peak; +- if `β=0.98` it would be 50 days. + +Here 10 or 50 days is called one lifetime (1 *e*-folding). Generally, for an exponential decay quantity, after one lifetime (`1/(1-β)` iterations), `1/e ≈ 37%` is remained and after two lifetime, `1/e^2 ≈ 14%` is left. + +For more information, check the definition of [*e*-folding](https://en.formulasearchengine.com/wiki/E-folding). + +#### Bias correction in exponentially weighted averages + +There's one technical detail called biased correction that can make you computation of these averages more accurately. In the temperature example above, when we set `β=0.98`, we won't actually get the green curve; instead, we get the purple curve (see the graph below). + +![ewa3](../_resources/ewa-temp-plot3.png) + +Because when we're implementing the exponentially weighted moving average, we initialize it with `V0=0`, subsequently we have the following result in the beginning of the iteration: + +- `V1 = 0.98*V0 + 0.02*θ1 = 0.02 * θ1` +- `V2 = 0.98*V1 + 0.02*θ2 = 0.0196 * θ1 + 0.02 * θ2` + +As a result, `V1` and `V2` calculated by this are not very good estimates of the first two temperature. So we need some modification to make it more accurate, especially during the initial phase of our estimate to avoid an __initial bias__. This can be corrected by scaling with `1/(1-β^t)` where `t` is the iteration number. + +| original | correction | +| --- | --- | +| ![V1](../_resources/bias-c1.svg) | ![V1c](../_resources/bias-c2.svg) | +| ![V2](../_resources/bias-c3.svg) | ![V2c](../_resources/bias-c4.svg) | + +#### Gradient descent with momentum + +Because mini-batch gradient descent makes a parameter update after seeing just a subset of examples, the direction of the update has some variance, and so the path taken by mini-batch gradient descent will "oscillate" toward convergence. Using momentum can reduce these oscillations. + +- gradient descent with momentum, which computes an EWA of gradients to update weights almost always works faster than the standard gradient descent algorithm. +- algorithm has two hyperparameters of `alpha`, the learning rate, and `beta` which controls your exponentially weighted average. common value for `beta` is `0.9`. +- don't bother with bias correction + +![momentum-algo](../_resources/momentum-algo.png) + +*Implementation tips*: + +- If `β = 0`, then this just becomes standard gradient descent without momentum. +- The larger the momentum `β` is, the smoother the update because the more we take the past gradients into account. But if `β` is too big, it could also smooth out the updates too much. +- Common values for `β` range from `0.8` to `0.999`. If you don't feel inclined to tune this, `β = 0.9` is often a reasonable default. +- It can be applied with batch gradient descent, mini-batch gradient descent or stochastic gradient descent. + +#### RMSprop + +RMSprop(root mean square), similar to momentum, has the effects of damping out the oscillations in gradient descent and mini-batch gradient descent and allowing you to maybe use a larger learning rate alpha. + +The algorithm computes the exponentially weighted averages of the squared gradients and updates weights by the square root of the EWA. + +``` +for iteration t: + # compute dW, db on mini-batch + + S_dW = (beta * S_dW) + (1 - beta) * dW^2 + S_db = (beta * S_db) + (1 - beta) * db^2 + W = W - alpha * dW / sqrt(S_dW + 𝜀) # 𝜀: small number(10^-8) to avoid dividing by zero + b = b - alpha * db / sqrt(S_db + 𝜀) +``` + +#### Adam optimization algorithm + +- Adam (Adaptive Moment Estimation) optimization algorithm is basically putting momentum and RMSprop together and combines the effect of gradient descent with momentum together with gradient descent with RMSprop. +- This is a commonly used learning algorithm that is proven to be very effective for many different neural networks of a very wide variety of architectures. +- In the typical implementation of Adam, bias correction is on. + +``` +V_dW = 0 +V_db = 0 +S_dW = 0 +S_db = 0 + +for iteration t: + # compute dW, db using mini-batch + + # momentum + V_dW = (beta1 * V_dW) + (1 - beta1) * dW + V_db = (beta1 * V_db) + (1 - beta1) * db + + # RMSprop + S_dW = (beta2 * S_dW) + (1 - beta2) * dW^2 + S_db = (beta2 * S_db) + (1 - beta2) * db^2 + + # bias correction + V_dW_c = V_dW / (1 - beta1^t) + V_db_c = V_db / (1 - beta1^t) + S_dW_c = S_dW / (1 - beta2^t) + S_db_c = S_db / (1 - beta2^t) + + W = W - alpha * V_dW_c / (sqrt(S_dW_c) + 𝜀) + b = b - alpha * V_db_c / (sqrt(S_db_c) + 𝜀) +``` + +*Implementation tips*: + +1. It calculates an exponentially weighted average of past gradients, and stores it in variables `V_dW,V_db` (before bias correction) and `V_dW_c,V_db_c` (with bias correction). +2. It calculates an exponentially weighted average of the squares of the past gradients, and stores it in variables `S_dW,S_db` (before bias correction) and `S_dW_c,S_db_c` (with bias correction). +3. It updates parameters in a direction based on combining information from "1" and "2". + +| hyperparameter | guideline | +| --- | --- | +| `learning rate` | tune | +| `beta1` (parameter of the momentum, for `dW`) | `0.9` | +| `beta2` (parameter of the RMSprop, for `dW^2`) | `0.999` | +| `𝜀` (avoid dividing by zero) | `10^-8` | + +Adam paper: [Adam: A Method for Stochastic Optimization](https://arxiv.org/abs/1412.6980) + +#### Learning rate decay + +The learning algorithm might just end up wandering around, and never really converge, because you're using some fixed value for alpha. Learning rate decay methods can help by making learning rate smaller when optimum is near. There are several decay methods: + +| decay factor | description | +| --- | --- | +| `0.95^epoch_num` | exponential decay | +| `k/sqrt(epoch_num)` or `k/sqrt(t)` | polynomial decay | +| discrete staircase | piecewise constant | +| manual decay | -- | + +#### The problem of local optima + +- First, you're actually pretty unlikely to get stuck in bad local optima, but much more likely to run into a saddle point, so long as you're training a reasonably large neural network, save a lot of parameters, and the cost function J is defined over a __relatively high dimensional space__. +- Second, that plateaus are a problem and you can actually make learning pretty slow. And this is where algorithms like __momentum__ or __RMSProp__ or __Adam__ can really help your learning algorithm. + +This is what a saddle point look like. + +![saddle-point](../_resources/saddle-point.png) + +#### Quick notes for optimization algorithms + +Recall that in [Course 1](joplin://8b8d24c8270944829c58a2071481e8b7#building-blocks-of-deep-neural-networks) we have already known that there are several steps in the neural network implementation: + +1. Initialize parameters / Define hyperparameters +2. Loop for num_iterations: + 1. Forward propagation + 2. Compute cost function + 3. Backward propagation + 4. __Update parameters (using parameters, and grads from backprop)__ +3. Use trained parameters to predict labels + +When we create `momentum`, `RMSprop` or `Adam` optimization methods, what we do is to implement algorithms in the __update parameters__ step. A good practice is to wrap them up as options so we can compare them during our alchemy training： + +``` +if optimizer == "gd": + parameters = update_parameters_with_gd(parameters, grads, learning_rate) +elif optimizer == "momentum": + parameters, v = update_parameters_with_momentum(parameters, grads, v, beta, learning_rate) +elif optimizer == "adam": + t = t + 1 # Adam counter + parameters, v, s = update_parameters_with_adam(parameters, grads, v, s, t, learning_rate, beta1, beta2, epsilon) +``` + +## Week 3: Hyperparameter tuning, Batch Normalization and Programming Frameworks + +### Learning Objectives + +- Master the process of hyperparameter tuning + +### Hyperparameter tuning + +#### Tuning process + +Importance of hyperparameters (roughly): + +| importance level | hyperparameters | +| --- | --- | +| first | learning rate `alpha` | +| second | momentum term `beta`
mini-batch size
number of hidden units | +| third | number of layers
learning rate decay
Adam `beta1, beta2, epsilon` | + +*Tuning tips*: + +- Choose points at random, not in a grid +- Optionally use a coarse to fine search process + +#### Using an appropriate scale to pick hyperparameters + +Search for hyperparameters on a log scale. + +``` +r = -4 * np.random.rand() # r in [-4,0] +alpha = 10**r # alpha in [10^-4, 1] +``` + +It's easy to extend to a more generalized case `[a,b]`. + +As for `beta`, use the same logarithmic scale method for `1-beta`. + +#### Hyperparameters tuning in practice: Panda vs. Caviar + +- __Panda approach__: Not enough computational capacity: babysitting one model +- __Caviar approach__: training many models in parallel + +### Batch Normalization + +#### Normalizing activations in a network + +- Batch normalization makes your hyperparameter search problem much easier, makes your neural network much more robust. +- What batch norm does is it applies that normalization process not just to the input layer, but to the values even deep in some hidden layer in the neural network. So it will apply this type of normalization to normalize the mean and + variance of `z[i]` of hidden units. +- One difference between the training input and these hidden unit values is that you might not want your hidden unit values be forced to have mean 0 and variance 1. + - For example, if you have a sigmoid activation function, you don't want your values to always be clustered in the normal distribution around `0`. You might want them to have a larger variance or have a mean that's different than 0, in order to better take advantage of the nonlinearity of the sigmoid function rather than have all your values be in just this linear region (near `0` on sigmoid function). + - What it does really is it then shows that your hidden units have standardized mean and variance, where the mean and variance are controlled by two explicit parameters `gamma` and `beta` which the learning algorithm can set to whatever it wants. + +![batch-norm](../_resources/batch-norm.png) + +#### Fitting Batch Norm into a neural network + +- `𝛽[1],𝛾[1],𝛽[2],𝛾[2],⋯,𝛽[𝐿],𝛾[𝐿]` can also be updated using gradient descent with momentum (or RMSprop, Adam). `𝛽[l],𝛾[l]` have the shape with `z[l]`. +- Similar computation can also be applied to mini-batches. +- With batch normalization, the parameter `b[l]` can be eliminated. So `w[l],𝛽[l],𝛾[l]` need to be trained. +- The parameter `𝛽` here has nothing to do with the `beta` in the momentum, RMSprop or Adam algorithms. + +![batch-norm-nn](../_resources/batch-norm-nn.png) + +#### Why does Batch Norm work + +- By normalizing all the features, input features `X`, to take on a similar range of values that can speed up learning. So batch normalization is doing a similar thing. +- To avoid *covariate shift* of data distribution, which makes the parameters change a lot in the training progress. Batch norm can reduce the amount that the distribution of the hidden unit values shifts around by making the mean and variance of `z` values remain the same. + - It allows each layer of the network to learn by itself, a little bit more independently of other layers, and this has the effect of speedup of learning in the whole network. + - From the perspective of one of the later layers of the neural network, the earlier layers don't get to shift around as much, because they're constrained to have the same mean and variance. This makes the job of learning on the later layers easier. +- It has a slight regularization effect. + - The mean and variance is a little bit noisy because it's estimated with just a relatively small sample of data (each mini-batch). So similar to dropout, it adds some noise to each hidden layer's activations. + - It's forcing the downstream hidden units not to rely too much on any one hidden unit. + - The noise added is quite small, so not a huge regularization effect. You can use batch norm together with dropouts if you want the more powerful regularization effect of dropout. + - Using bigger mini-batch size can reduce noise and therefore reduce regularization effect. + - Don't turn to batch norm as a regularization. This is not the intent of batch norm. + - Just use it as a way to normalize hidden units activations and therefore speedup learning. +- At test time, you try to make predictors and evaluate the neural network, you might not have a mini-batch of examples, you might be processing one single example at the time. So, at test time you need to do something slightly differently to make sure your predictions make sense. + +#### Batch Norm at test time + +- Batch norm processes our data one mini batch at a time, but at test time we need to process examples one at a time. +- In theory we could run the whole training set through final network to get `𝜇` and `𝜎^2`. +- In practice, usually implement an exponentially weighted average where we just keep track of the `𝜇` and `𝜎^2` we're seeing during training and use an EWA (across mini-batches), also sometimes called the running average, to get a rough estimate of `𝜇` and `𝜎^2` and then use these to scale at test time. + - `𝜇{1}[l], 𝜇{2}[l], 𝜇{3}[l], ...` —\> `𝜇[l]` + - `𝜎^2{1}[l], 𝜎^2{2}[l], 𝜎^2{3}[l], ...` —\> `𝜎^2[l]` + +### Multi-class classification + +#### Softmax Regression + +Use softmax activation function. + +``` +def softmax(z): + return np.exp(z) / sum(np.exp(z)) + +z = [1,0.5,-2,1,3] +print(softmax(z)) +# array([0.09954831, 0.0603791 , 0.00495622, 0.09954831, 0.73556806]) +``` + +#### Training a softmax classifier + +Softmax regression is a generalization of logistic regression to more than two classes. + +### Introduction to programming frameworks + +#### Deep learning frameworks + +- Caffe/Caffe2 +- CNTK +- DL4J +- Keras +- Lasagne +- mxnet +- PaddlePaddle +- TensorFlow +- Theano +- Torch + +*Choosing deep learning frameworks*: + +- Ease of programming (development and deployment) +- Running speed +- Truly open (open source with good governance) + +#### Tensorflow + +- The two main object classes in tensorflow are *Tensors* and *Operators*. +- When we code in tensorflow we have to take the following steps: + - Create a graph containing Tensors (*Variables*, *Placeholders* ...) and *Operations* (`tf.matmul`, `tf.add`, ...) + - Create a *session* + - Initialize the *session* + - Run the *session* to execute the graph +- We might need to execute the graph multiple times when implementing `model()` +- The backpropagation and optimization is automatically done when running the session on the "optimizer" object. + +``` +import numpy as np  +import tensorflow as tf + +coefficients = np.array([[1], [-20], [25]]) +w = tf.Variable([0],dtype=tf.float32) +x = tf.placeholder(tf.float32, [3,1]) +cost = x[0][0]*w**2 + x[1][0]*w + x[2][0]    # (w-5)**2 +train = tf.train.GradientDescentOptimizer(0.01).minimize(cost) +init = tf.global_variables_initializer() +session = tf.Session() +session.run(init)  +print(session.run(w)) + +for i in range(1000): + session.run(train, feed_dict={x:coefficients}) +print(session.run(w)) +``` + +* * * + +Notes by Aaron © 2020 diff --git a/DeepLearning_AI/Course 3_ Structuring Machine Learning Projects.md b/DeepLearning_AI/Course 3_ Structuring Machine Learning Projects.md new file mode 100644 index 0000000..56ae107 --- /dev/null +++ b/DeepLearning_AI/Course 3_ Structuring Machine Learning Projects.md @@ -0,0 +1,382 @@ +--- +title: 'Course 3: Structuring Machine Learning Projects' +updated: 2022-05-17 18:57:41Z +created: 2022-05-16 17:54:31Z +--- + +# Course 3: Structuring Machine Learning Projects + +- [Course 3: Structuring Machine Learning Projects](#course-3-structuring-machine-learning-projects) + - [Week 1: ML Strategy (1)](#week-1-ml-strategy-1) + - [Learning Objectives](#learning-objectives) + - [Introduction to ML Strategy](#introduction-to-ml-strategy) + - [Why ML Strategy](#why-ml-strategy) + - [Orthogonalization](#orthogonalization) + - [Setting up your goal](#setting-up-your-goal) + - [Single number evaluation metric](#single-number-evaluation-metric) + - [Satisficing and optimizing metric](#satisficing-and-optimizing-metric) + - [Train/dev/test distributions](#traindevtest-distributions) + - [Size of the dev and test sets](#size-of-the-dev-and-test-sets) + - [When to change dev/test sets and metrics](#when-to-change-devtest-sets-and-metrics) + - [Comparing to human-level performance](#comparing-to-human-level-performance) + - [Why human-level performance](#why-human-level-performance) + - [Avoidable bias](#avoidable-bias) + - [Understanding human-level performance](#understanding-human-level-performance) + - [Surpassing human-level performance](#surpassing-human-level-performance) + - [Improving your model performance](#improving-your-model-performance) + - [Week 2: ML Strategy (2)](#week-2-ml-strategy-2) + - [Learning Objectives](#learning-objectives-1) + - [Error Analysis](#error-analysis) + - [Carrying out error analysis](#carrying-out-error-analysis) + - [Cleaning up incorrectly labeled data](#cleaning-up-incorrectly-labeled-data) + - [Build your first system quickly, then iterate](#build-your-first-system-quickly-then-iterate) + - [Mismatched training and dev/test set](#mismatched-training-and-devtest-set) + - [Training and testing on different distributions](#training-and-testing-on-different-distributions) + - [Bias and Variance with mismatched data distributions](#bias-and-variance-with-mismatched-data-distributions) + - [Addressing data mismatch](#addressing-data-mismatch) + - [Learning from multiple tasks](#learning-from-multiple-tasks) + - [Transfering learning](#transfering-learning) + - [Multi-task learning](#multi-task-learning) + - [End-to-end deep learning](#end-to-end-deep-learning) + - [What is end-to-end deep learning](#what-is-end-to-end-deep-learning) + - [Whether to use end-to-end deep learning](#whether-to-use-end-to-end-deep-learning) + +## Week 1: ML Strategy (1) + +### Learning Objectives + +- Explain why Machine Learning strategy is important +- Apply satisficing and optimizing metrics to set up your goal for ML projects +- Choose a correct train/dev/test split of your dataset +- Define human-level performance +- Use human-level performance to define key priorities in ML projects +- Take the correct ML Strategic decision based on observations of performances and dataset + +### Introduction to ML Strategy + +#### Why ML Strategy + +*Ideas to improve a machine learning system*: + +- Collect more data +- Collect more diverse training set +- Train algorithm longer with gradient descent +- Try Adam instead of gradient descent +- Try bigger network +- Try smaller network +- Try dropout +- Add L2 regularization +- Network architecture + - Activation functions + - number of hidden units + - ... + +In order to have quick and effective ways to figure out which of all of these ideas and maybe even other ideas, are worth pursuing and which ones we can safely discard, we need ML strategies. + +#### Orthogonalization + +In the example of TV tuning knobs, orthogonalization refers to that the TV designers had designed the knobs so that each knob kind of does only one thing. + +In a car the stearing wheel controls the angle and the accelerator and brake control the speed. If there are two controllers, each has different effect simultaneously on angle and speed, then it's much harder to set the car to the speed and angle we want. + +``` +0.3 * angle - 0.8 * speed +2 * angle + 0.9 * speed +``` + +Orthogonal means at 90 degrees to each other. By having orthogonal controls that are ideally aligned with the things we actually want to control. It makes it much easier to tune the knobs we have to tune. To tune the steering wheel angle, and the accelerator, the brake, to get the car to do what we want. + +| chain of assumptions in ML | tune the *knobs* | +| --- | --- | +| Fit training set well on cost function | bigger network
better optimization algorithm, Adam... | +| Fit dev set well on cost function | regularization
bigger training set | +| Fit test set well on cost function | bigger dev set | +| Performs well in real world | change dev set or cost function
(dev test set distribution not correct or cost function not right) | + +Early stopping, though not a bad technique, is a *knob* that simultaneously affects the training set and dev set performance, and therefore is **less orthogonalized**, so Andrew tend not to use it. + +### Setting up your goal + +#### Single number evaluation metric + +Evaluation metric allows you to quickly tell if classifier A or classifier B is better, and therefore having a dev set plus single number evaluation metric tends to speed up iterating. + +| metric | calculation | definition | +| --- | --- | --- | +| Precision | `P = TP/(TP+FP)` | percentage of true positive in predicted positive | +| Recall | `R = TP/(TP+FN)` | percentage of true positive predicted in all real positive | +| F1 score | `F1 = 2PR/(P+R)` or `1/F1 = (1/P+1/R)/2` | harmonic mean of precision and recall | + +#### Satisficing and optimizing metric + +If we care about the classification accuracy of our cat's classifier and also care about the running time or some other performance, instead of combining them into an overall evaluation metric by their *artificial* linear weighted sum, we actually can have one thing as an **optimizing metric** and the others as **satisficing metrics**. + +- In the cat's classifier example, we might have accuracy as optimizing metric and running time as satificing metric. +- In wake words detection system (like Amazon Echo, Apple Siri, ...), maybe accuracy is an optimizing metric and false positive `≤ 1` every 24 hours is a satificing metric. + +#### Train/dev/test distributions + +*Guideline*: + +- Choose a dev set and test set to reflect data you expect to get in future and consider important to do well on. +- In particular, **the dev set and the test set here, should come from the same distribution**. + +#### Size of the dev and test sets + +- In the era of big data, the old rule of thumb of a 70/30 is that, that no longer applies. And the trend has been to use more data for training and less for dev and test, especially when you have a very large data sets. +- Suppose we have a million training examples, it might be quite reasonable to set up the data so that we have 98% in the training set, 1% dev, and 1% test. +- The guideline is, to set your test set to big enough to give high confidence in the overall performance of your system. +- When people were talking about using train test splits, what they actually had was a train dev split and no test set. +- In the history of machine learning, not everyone has been completely clean and completely records of about calling the dev set when it really should be treated as dev set. + +#### When to change dev/test sets and metrics + +In an example of cat classification system, classification error might not be a reasonable metric if two algorithms have the following performance: + +| algorithm | classification error | issues | review | +| --- | --- | --- | --- | +| Algorithm A | 3% | letting through lots of porn images | showing pornographic images to users is intolerable | +| Algorithm B | 5% | no pornographic images | classifies fewer images but acceptable | + +In this case, metric should be modified. One way to change this evaluation metric would be adding weight terms. + +| metric | calculation | notation | +| --- | --- | --- | +| classification error | ![clf-error](../_resources/metric-clf-error.svg) | `L` can be identity function to count correct labels | +| weighted classification error | ![clf-error-weighted](../_resources/metric-clf-error-weighted.svg) | ![weights](../_resources/metric-clf-error-weights.svg) | + +So if you find that evaluation metric is not giving the correct rank order preference for what is actually better algorithm, then there's a time to think about defining a new evaluation metric. + +This is actually an example of an orthogonalization where I think you should take a machine learning problem and break it into distinct steps. + +- First, figure out how to define a metric that captures what you want to do. (*place the target*) +- Second, think about how to actually do well on this metric. (*shoot the target*) + +The overall guideline is if your current metric and data you are evaluating on doesn't correspond to doing well on what you actually care about, then change your metrics and/or your dev/test set to better capture what you need your algorithm to actually do well on. + +### Comparing to human-level performance + +#### Why human-level performance + +A lot more machine learning teams have been talking about comparing the machine learning systems to human-level performance. + +- First, because of advances in deep learning, machine learning algorithms are suddenly working much better and so it has become much more feasible in a lot of application areas for machine learning algorithms to actually become competitive with human-level performance. +- Second, the workflow of designing and building a machine learning system is much more efficient when we're trying to do something that humans can also do. + +The graph below shows the performance of humans and machine learning over time. + +![human-performance](../_resources/human-performance.png) + +Machine learning progresses slowly when it surpasses human-level performance. One of the reason is that human-level performance can be closeto Bayes optimal error, especially for natural perception problem. + +Bayes optimal error is defined as the best possible error. In other words, it means that any functions mapping from x to y can’t surpass a certain level of accuracy. + +Also, when the performance of machine learning is worse than the performance of humans, we can improve it with different tools. They are harder to use once it surpasses human-level performance. + +*These tools are*: + +- Get labelled data from humans +- Gain insight from manual error analysis: Why did a person get this right? +- Better analysis of bias/variance. + +#### Avoidable bias + +By knowing what the human-level performanceis, it is possible to tell when a training set is performing well or not. + +| performance | Scenario A | Scenario B | +| --- | --- | --- | +| humans | 1 | 7.5 | +| training error | 8 | 8 | +| development error | 10 | 10 | + +In this case, the human-level error as a proxy for Bayes error since humans are good to identify images. If you want to improve the performance of the training set but you can’t do better than the Bayes error otherwise the training set is overfitting. By knowing the Bayes error, it is easier to focus on whether bias or variance avoidance tactics will improve the performance of the model. + +- *Scenario A*: There is a 7% gap between the performance of the training set and the human-level error. It means that the algorithm isn’t fitting well with the training set since the target is around 1%. To resolve the issue, we use bias reduction technique such as training a bigger neural network or running the training set longer. + +- *Scenario B*: The training set is doing good since there is only a 0.5% difference with the human-level error. The difference between the training set and the human-level error is called **avoidable bias**. The focus here is to reduce the variance since the difference between the training error and the development error is 2%. To resolve the issue, we use variance reduction technique such as regularization or have a bigger training set. + + +#### Understanding human-level performance + +Summary of bias/variance with human-level performance: + +- Human-level error is a proxy for Bayes error. +- If the difference between human-level error and the training error is bigger than the difference between the training error and the development error. The focus should be on bias reduction technique. +- If the difference between training error and the development error is bigger than the difference between the human-level error and the training error. The focus should be on variance reduction technique + +#### Surpassing human-level performance + +*Classification task performance (classification error)*: + +| performance | Scenario A | Scenario B | +| --- | --- | --- | +| Team of humans | 0.5 | 0.5 | +| One human | 1.0 | 1 | +| Training error | 0.6 | 0.3 | +| Development error | 0.8 | 0.4 | + +- Scenario A: In this case, the Bayes error is 0.5%, therefore the available bias is 0.1%et the variance is 0.2%. +- Scenario B: In this case, there is not enough information to know if bias reduction or variance reduction has to be done on the algorithm. It doesn’t mean that the model cannot be improve, it means that the conventional ways to know if bias reduction or variance reduction are not working in this case. + +There are many problems where machine learning significantly surpasses human-level performance, especially with structured data: + +| problem | structured data | +| --- | --- | +| Online advertising | database of what has users clicked on | +| Product recommendations | database of proper support for | +| Logistics (predicting transit time) | database of how long it takes to get from A to B | +| Loan approvals | database of previous loan applications and their outcomes | + +And these are not **natural perception problems**, so these are not *computer vision*, or *speech recognition*, or *natural language processing* task. Humans tend to be very good in natural perception task. So it is possible, but it's just a bit harder for computers to surpass human-level performance on natural perception task. + +#### Improving your model performance + +*There are two fundamental assumptions of supervised learning.* + +- The first one is to have a low avoidable bias which means that the training set fits well. +- The second one is to have a low or acceptable variance which means that the training set performance generalizes well to the development set and test set. + +![improve-model-performance](../_resources/improve-performance.png) + +## Week 2: ML Strategy (2) + +### Learning Objectives + +- Describe multi-task learning and transfer learning +- Recognize bias, variance and data-mismatch by looking at the performances of your algorithm on train/dev/test sets + +### Error Analysis + +#### Carrying out error analysis + +*To carry out error analysis, you should*: + +- find a set of mislabeled examples in your dev set. +- look at the mislabeled examples for false positives and false negatives. +- count up the number of errors that fall into various different categories. +- you might be inspired to generate new categories of errors. + +#### Cleaning up incorrectly labeled data + +*Some facts*: + +- Deep learning algorithms are quite robust to random errors in the training set. +- The goal of the dev set, the main purpose of the dev set is, you want to really use it to help you select between two classifiers A and B. +- It's super important that your dev and test sets come from the same distribution. + +*Correcting incorrect dev/test set examples*: + +- Apply same process to your dev and test sets to make sure they continue to come from the same distribution. +- Consider examining examples your algorithm got right as well as ones it got wrong. +- Train and dev/test data may now come from slightly different distributions. + +#### Build your first system quickly, then iterate + +Depending on the area of application, the guideline below will help you prioritize when you build your system. + +*Guideline*: + +1. Set up development/test set and metrics + 1. Set up a target +2. Build an initial system quickly + 1. Train training set quickly: Fit the parameters + 2. Development set: Tune the parameters + 3. Test set: Assess the performance +3. Use **bias/variance analysis** & **error analysis** to prioritize next steps + +### Mismatched training and dev/test set + +#### Training and testing on different distributions + +In the *Cat vs Non-cat* example, there are two sources of data used to develop **the mobile app**. + +- The first data distribution is small, 10,000 pictures uploaded from the mobile application. Since they are from amateur users,the pictures are not professionally shot, not well framed and blurrier. +- The second source is from the web, you downloaded 200,000 pictures where cat’s pictures are professionally framed and in high resolution. + +The guideline is that you have to choose a development set and test set to reflect data you expect to get **in the future** and consider important to do well. + +![data-on-diff-dist](../_resources/data-dist.png) + +#### Bias and Variance with mismatched data distributions + +Instead of just having bias and variance as two potential problems, you now have a third potential problem, data mismatch. + +![bias-variance-mismatched](../_resources/bias-variance-mismatch.png) + +![bias-variance-mismatched-1](../_resources/bias-variance-mismatch-1.png) + +#### Addressing data mismatch + +*This is a general guideline to address data mismatch*: + +- Perform manual error analysis to understand the error differences between training, development/test sets. Development should never be done on test set to avoid overfitting. +- Make training data or collect data similar to development and test sets. To make the training data more similar to your development set, you can use is **artificial data synthesis**. However, it is possible that if you might be accidentally simulating data only from a tiny subset of the space of all possible examples. + +### Learning from multiple tasks + +#### Transfering learning + +Transfer learning refers to using the neural network knowledge for another application. + +*When to use transfer learning*: + +- Task A and B have the same input 𝑥 +- A lot more data for Task A than Task B +- Low level features from Task A could be helpful for Task B + +*Example 1: Cat recognition - radiology diagnosis* + +The following neural network is trained for cat recognition, but we want to adapt it for radiology diagnosis. The neural network will learn about the structure and the nature of images. This initial phase of training on image recognition is called **pre-training**, since it will pre-initialize the weights of the neural network. Updating all the weights afterwards is called **fine-tuning**. + +*Guideline*: + +- Delete last layer of neural network +- Delete weights feeding into the last output layer of the neural network +- Create a new set of randomly initialized weights for the last layer only +- New data set `(𝑥, 𝑦)` + +![transfer-learning](../_resources/transfer-learning.png) + +#### Multi-task learning + +Multi-task learning refers to having one neural network do simultaneously several tasks. + +*When to use multi-tasklearning*: + +- Training on a set of tasks that could benefit from having shared lower-level features +- Usually: Amount of data youhave for each task is quite similar +- Can train a big enough neural network to do well on all tasks + +![multi-task](../_resources/multi-task-learning.png) + +### End-to-end deep learning + +#### What is end-to-end deep learning + +- End-to-end deep learning is the simplification of a processing or learning systems into one neural network. +- End-to-end deep learning cannot be used for every problem since it needs a lot of labeled data. It is used mainly in + - audio transcripts, + - image captures, + - image synthesis, + - machine translation, + - steering in self-driving cars, etc. + +![end-to-end](../_resources/end-to-end.png) + +#### Whether to use end-to-end deep learning + +Before applying end-to-end deep learning, you need to ask yourself the following question: Do you have enough data to learn a function of the complexity needed to map x and y? + +*Pro*: + +- *Let the data speak*. By having a pure machine learning approach, the neural network will learn from x to y. It will be able to find which statistics are in the data, rather than being forced to reflect human preconceptions. +- *Less hand-designing of components needed*. It simplifies the design work flow. + +*Cons*: + +- *Large amount of labeled data*. It cannot be used for every problem as it needs a lot of labeled data. +- *Excludes potentially useful hand-designed component*. Data and any hand-design’s components or features are the 2 main sources of knowledge for a learning algorithm. If the data set is small than a hand-design system is a way to give manual knowledge into the algorithm. + +* * * + +Notes by Aaron © 2020 \ No newline at end of file diff --git a/DeepLearning_AI/Course 4_ Convolutional Neural Networks.md b/DeepLearning_AI/Course 4_ Convolutional Neural Networks.md new file mode 100644 index 0000000..12043ec --- /dev/null +++ b/DeepLearning_AI/Course 4_ Convolutional Neural Networks.md @@ -0,0 +1,818 @@ +--- +title: 'Course 4: Convolutional Neural Networks' +updated: 2022-05-23 16:40:37Z +created: 2022-05-16 17:54:54Z +--- + +# Course 4: Convolutional Neural Networks + +- [Course 4: Convolutional Neural Networks](#course-4-convolutional-neural-networks) + - [Week 1: Foundations of Convolutional Neural Networks](#week-1-foundations-of-convolutional-neural-networks) + - [Learning Objectives](#learning-objectives) + - [Convolutional Neural Networks](#convolutional-neural-networks) + - [Computer Vision](#computer-vision) + - [Edge Detection Example](#edge-detection-example) + - [More Edge Detection](#more-edge-detection) + - [Padding](#padding) + - [Strided Convolutions](#strided-convolutions) + - [Convolutions Over Volume](#convolutions-over-volume) + - [One Layer of a Convolutional Network](#one-layer-of-a-convolutional-network) + - [Simple Convolutional Network](#simple-convolutional-network) + - [Pooling Layers](#pooling-layers) + - [CNN Example](#cnn-example) + - [Why Convolutions](#why-convolutions) + - [Week 2: Classic Networks](#week-2-classic-networks) + - [Learning Objectives](#learning-objectives-1) + - [Case Studies](#case-studies) + - [Why look at case studies](#why-look-at-case-studies) + - [Classic Networks](#classic-networks) + - [LeNet-5](#lenet-5) + - [AlexNet](#alexnet) + - [VGG-16](#vgg-16) + - [ResNets](#resnets) + - [Why ResNets](#why-resnets) + - [Networks in Networks and 1x1 Convolutions](#networks-in-networks-and-1x1-convolutions) + - [Inception Network Motivation](#inception-network-motivation) + - [Inception Network](#inception-network) + - [Practical advices for using ConvNets](#practical-advices-for-using-convnets) + - [Using Open-Source Implementation](#using-open-source-implementation) + - [Transfering Learning](#transfering-learning) + - [Data Augmentation](#data-augmentation) + - [State of Computer Vision](#state-of-computer-vision) + - [Tips for Keras](#tips-for-keras) + - [Week 3: Object detection](#week-3-object-detection) + - [Learning Objectives](#learning-objectives-2) + - [Detection algorithms](#detection-algorithms) + - [Object Localization](#object-localization) + - [Landmark Detection](#landmark-detection) + - [Object Detection](#object-detection) + - [Convolutional Implementation of Sliding Windows](#convolutional-implementation-of-sliding-windows) + - [Bounding Box Predictions (YOLO)](#bounding-box-predictions-yolo) + - [Intersection Over Union](#intersection-over-union) + - [Non-max Suppression](#non-max-suppression) + - [Anchor Boxes](#anchor-boxes) + - [YOLO Algorithm](#yolo-algorithm) + - [(Optional) Region Proposals](#optional-region-proposals) + - [Week 4: Special applications: Face recognition & Neural style transfer](#week-4-special-applications-face-recognition--neural-style-transfer) + - [Face Recognition](#face-recognition) + - [What is face recognition](#what-is-face-recognition) + - [One Shot Learning](#one-shot-learning) + - [Siamese network](#siamese-network) + - [Triplet Loss](#triplet-loss) + - [Face Verification and Binary Classification](#face-verification-and-binary-classification) + - [Summary of Face Recognition](#summary-of-face-recognition) + - [Neural Style Transfer](#neural-style-transfer) + - [What is neural style transfer](#what-is-neural-style-transfer) + - [What are deep ConvNets learning](#what-are-deep-convnets-learning) + - [Cost Function](#cost-function) + - [Content Cost Function](#content-cost-function) + - [Style Cost Function](#style-cost-function) + - [1D and 3D Generalizations](#1d-and-3d-generalizations) + +## Week 1: Foundations of Convolutional Neural Networks + +### Learning Objectives + +- Explain the convolution operation +- Apply two different types of pooling operations +- Identify the components used in a convolutional neural network (padding, stride, filter, ...) and their purpose +- Build and train a ConvNet in TensorFlow for a classification problem + +### Convolutional Neural Networks + +#### Computer Vision + +*Deep learning computer vision can now*: + +- help self-driving cars figure out where the other cars and pedestrians around so as to avoid them. +- make face recognition work much better than ever before. +- unlock a phone or unlock a door using just your face. + +*Deep learning for computer vision is exciting* because: + +- First, rapid advances in computer vision are enabling brand new applications to view, though they just were impossible a few years ago. +- Second, even if you don't end up building computer vision systems per se, I found that because the computer vision research community has been so creative and so inventive in coming up with new neural network architectures and algorithms, is actually inspire that creates a lot cross-fertilization into other areas as well. + +For computer vision applications, you don't want to be stuck using only tiny little images. You want to use large images. To do that, you need to better implement the **convolution operation**, which is one of the fundamental building blocks of **convolutional neural networks**. + +#### Edge Detection Example + +- The convolution operation is one of the fundamental building blocks of a convolutional neural network. +- Early layers of the neural network might detect edges and then some later layers might detect parts of objects and then even later layers may detect parts of complete objects like people's faces. +- Given a picture for a computer to figure out what are the objects in the picture, the first thing you might do is maybe detect edges in the image. + +The *convolution operation* gives you a convenient way to specify how to find these **vertical edges** in an image. + +A `3 by 3` filter or `3 by 3` matrix may look like below, and this is called a vertical edge detector or a vertical edge detection filter. In this matrix, pixels are relatively bright on the left part and relatively dark on the right part. + +``` +1, 0, -1 +1, 0, -1 +1, 0, -1 +``` + +Convolving it with the vertical edge detection filter results in detecting the vertical edge down the middle of the image. + +![edge-detection](../_resources/edge-detect-v.png) + +#### More Edge Detection + +In the horizontal filter matrix below, pixels are relatively bright on the top part and relatively dark on the bottom part. + +``` + 1, 1, 1 + 0, 0, 0 +-1, -1, -1 +``` + +Different filters allow you to find vertical and horizontal edges. The following filter is called a **Sobel filter** the advantage of which is it puts a little bit more weight to the central row, the central pixel, and this makes it maybe a little bit more robust. [More about Sobel filter](https://fiveko.com/tutorials/image-processing/sobel-filter/). + +``` +1, 0, -1 +2, 0, -2 +1, 0, -1 +``` + +Here is another filter called **Scharr filter**: + +``` + 3, 0, -3 +10, 0, -10 + 3, 0, -3 +``` + +More about [**Scharr filter**](https://plantcv.readthedocs.io/en/v3.0.5/scharr_filter/). + +``` +w1, w2, w3 +w4, w5, w6 +w7, w8, w9 +``` + +By just letting all of these numbers be parameters and learning them automatically from data, we find that neural networks can actually learn low level features, can learn features such as edges, even more robustly than computer vision researchers are generally able to code up these things by hand. + +#### Padding + +In order to fix the following two problems, padding is usually applied in the convolutional operation. + +- Every time you apply a convolutional operator the image shrinks. +- A lot of information from the edges of the image is thrown away. + +*Notations*: + +- image size: `n x n` +- convolution size: `f x f` +- padding size: `p` + +*Output size after convolution*: + +- without padding: `(n-f+1) x (n-f+1)` +- with padding: `(n+2p-f+1) x (n+2p-f+1)` + +*Convention*: + +- Valid convolutions: no padding +- Same convolutions: output size is the same as the input size +- `f` is usually odd + +#### Strided Convolutions + +*Notation*: + +- stride `s` + +*Output size after convolution*: `floor((n+2p-f)/s+1) x floor((n+2p-f)/s+1)` + +*Conventions*: + +- The filter must lie entirely within the image or the image plus the padding region. +- In the deep learning literature by convention, a convolutional operation (maybe better *called cross-correlation*) is what we usually do not bother with a flipping operation, which is included before the product and summing step in a typical math textbook or a signal processing textbook. + - In the latter case, the filter is flipped vertically and horizontally. + +#### Convolutions Over Volume + +For a RGB image, the filter itself has three layers corresponding to the red, green, and blue channels. + +`height x width x channel` + +`n x n x nc` \* `f x f x nc` --\> `(n-f+1) x (n-f+1) x nc'` + +#### One Layer of a Convolutional Network + +*Notations*: + +| size | notation | +| --- | --- | +| filter size | ![f(l)](../_resources/layer_filter_size.svg) | +| padding size | ![p(l)](../_resources/layer_padding_size.svg) | +| stride size | ![s(l)](../_resources/layer_stride_size.svg) | +| number of filters | ![nc(l)](../_resources/layer_num_filters.svg) | +| filter shape | ![filter_shape](../_resources/layer_filter_shape.svg) | +| input shape | ![input_shape](../_resources/layer_input_shape.svg) | +| output shape | ![output_shape](../_resources/layer_output_shape.svg) | +| output height | ![nh(l)](../_resources/layer_output_height.svg) | +| output width | ![nw(l)](../_resources/layer_output_width.svg) | +| activations `a[l]` | ![activations](../_resources/layer_output_shape.svg) | +| activations `A[l]` | ![activations](../_resources/layer_activations.svg) | +| weights | ![weights](../_resources/layer_weights.svg) | +| bias | ![bias](../_resources/layer_bias.svg) | + +#### Simple Convolutional Network + +Types of layer in a convolutional network: + +- Convolution (CONV) +- Pooling (POOL) +- Fully connected (FC) + +#### Pooling Layers + +- One interesting property of max pooling is that it has a set of hyper-parameters but it has no parameters to learn. There's actually nothing for gradient descent to learn. +- Formulas that we had developed previously for figuring out the output size for conv layer also work for max pooling. +- The max pooling is used much more often than the average pooling. +- When you do max pooling, usually, you do not use any padding. +- why pooling: + - to allow a degree of translational invariance on the input. + - to down sample the spacial dimensions, thereby reducing the numbers of networks. + +#### CNN Example + +- Because the pooling layer has no weights, has no parameters, only a few hyper parameters, I'm going to use a convention that `CONV1` and `POOL1` shared together. +- As you go deeper usually the *height* and *width* will decrease, whereas the number of *channels* will increase. +- max pooling layers don't have any parameters +- The conv layers tend to have relatively few parameters and a lot of the parameters tend to be in the fully collected layers of the neural network. +- The activation size tends to maybe go down *gradually* as you go deeper in the neural network. If it drops too quickly, that's usually not great for performance as well. + +![nn-example](../_resources/nn-example.png) + +*Layer shapes of the network*: + +| layer | activation shape | activation size | \# parameters | +| --- | --- | --- | --- | +| Input | (32,32,3) | 3072 | 0 | +| CONV1 (f=5,s=1) | (28,28,8) | 6272 | 608 `=(5*5*3+1)*8` | +| *POOL1* | (14,14,8) | 1,568 | 0 | +| CONV2 (f=5,s=1) | (10,10,16) | 1600 | 3216 `=(5*5*8+1)*16` | +| *POOL2* | (5,5,16) | 400 | 0 | +| FC3 | (120,1) | 120 | 48120 `=400*120+120` | +| FC4 | (84,1) | 84 | 10164 `=120*84+84` | +| softmax | (10,1) | 10 | 850 `=84*10+10` | + +#### Why Convolutions + +There are two main advantages of convolutional layers over just using fully connected layers. + +- Parameter sharing: A feature detector (such as a vertical edge detector) that’s useful in one part of the image is probably useful in another part of the image. +- Sparsity of connections: In each layer, each output value depends only on a small number of inputs. + +Through these two mechanisms, a neural network has a lot fewer parameters which allows it to be trained with smaller training cells and is less prone to be overfitting. + +- Convolutional structure helps the neural network encode the fact that an image shifted a few pixels should result in pretty similar features and should probably be assigned the same output label. +- And the fact that you are applying the same filter in all the positions of the image, both in the early layers and in the late layers that helps a neural network automatically learn to be more robust or to better capture the desirable property of translation invariance. + +## Week 2: Classic Networks + +### Learning Objectives + +- Discuss multiple foundational papers written about convolutional neural networks +- Analyze the dimensionality reduction of a volume in a very deep network +- Implement the basic building blocks of ResNets in a deep neural network using Keras +- Train a state-of-the-art neural network for image classification +- Implement a skip connection in your network +- Clone a repository from github and use transfer learning + +### Case Studies + +#### Why look at case studies + +It is helpful in taking someone else's neural network architecture and applying that to another problem. + +- Classic networks + - LeNet-5 + - AlexNet + - VGG +- ResNet +- Inception + +#### Classic Networks + +##### LeNet-5 + +![LeNet-5](../_resources/lenet-5.png) + +Some difficult points about reading the [LeNet-5 paper](https://pdfs.semanticscholar.org/62d7/9ced441a6c78dfd161fb472c5769791192f6.pdf): + +- Back then, people used sigmoid and tanh nonlinearities, not relu. +- To save on computation as well as some parameters, the original LeNet-5 had some crazy complicated way where different filters would look at different channels of the input block. And so the paper talks about those details, but the more modern implementation wouldn't have that type of complexity these days. +- One last thing that was done back then I guess but isn't really done right now is that the original LeNet-5 had a non-linearity after pooling, and I think it actually uses sigmoid non-linearity after the pooling layer. +- Andrew Ng recommend focusing on section two which talks about this architecture, and take a quick look at section three which has a bunch of experiments and results, which is pretty interesting. Later sections talked about the graph transformer network, which isn't widely used today. + +##### AlexNet + +![AlexNet](../_resources/alexnet.png) + +- AlexNet has a lot of similarities to LeNet (60,000 parameters), but it is much bigger (60 million parameters). +- The paper had a complicated way of training on two GPUs since GPU was still a little bit slower back then. +- The original AlexNet architecture had another set of a layer called local response normalization, which isn't really used much. +- Before AlexNet, deep learning was starting to gain traction in speech recognition and a few other areas, but it was really just paper that convinced a lot of the computer vision community to take a serious look at deep learning, to convince them that deep learning really works in computer vision. + +##### VGG-16 + +![VGG-16](../_resources/vgg-16.png) + +- Filters are always `3x3` with a stride of `1` and are always `same` convolutions. +- VGG-16 has 16 layers that have weights. A total of about 138 million parameters. Pretty large even by modern standards. +- It is the simplicity, or the uniformity, of the VGG-16 architecture made it quite appealing. + - There is a few conv-layers followed by a pooling layer which reduces the height and width by a factor of `2`. + - Doubling through every stack of conv-layers is a simple principle used to design the architecture of this network. +- The main downside is that you have to train a large number of parameters. + +#### ResNets + +Paper: [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385) + +![resnet-network](../_resources/resnet-network.png) + +- Deeper neural networks are more difficult to train. They present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. +- When deeper networks are able to start converging, a degradation problem has been exposed: with the network depth increasing, accuracy gets saturated (which might be unsurprising) and then degrades rapidly. The paper address the degradation problem by introducing a deep residual learning framework. Instead of hoping each few stacked layers directly fit a desired underlying mapping, they explicitly let these layers fit a residual mapping. +- The paper authors show that: 1) Their extremely deep residual nets are easy to optimize, but the counterpart "plain" nets (that simply stack layers) exhibit higher training error when the depth increases; 2) Their deep residual nets can easily enjoy accuracy gains from greatly increased depth, producing results substantially better than previous networks. + +![resnet](../_resources/resnet.png) + +Formally, denoting the desired underlying mapping as `H(x)`, they let the stacked nonlinear layers fit another mapping of `F(x):=H(x)-x`. The original mapping `H(x)` is recast into `F(x)+x`. If the added layers can be constructed as identity mappings, a deeper model should have training error no greater than its shallower counterpart. + +![resnet-block](../_resources/resnet-block.png) + +#### Why ResNets + +- Doing well on the training set is usually a prerequisite to doing well on your hold up or on your depth or on your test sets. So, being able to at least train ResNet to do well on the training set is a good first step toward that. +- But if you make a network deeper, it can hurt your ability to train the network to do well on the training set. It is not true or at least less true when training a ResNet. + - If we use `L2` regularization on `a[l+2]=g(Z[l+2]+a[l])=g(W[l+2]a[l+1]+b[l+2]+a[l])`, and if the value of `W[l+2],b[l+2]` shrink to zero, then `a[l+2]=g(a[l])=a[l]` since we use `relu` activation and `a[l]` is also non-negative. So we just get back `a[l]`. This shows that the identity function is easy for residual block to learn. + - It's easy to get `a[l+2]` equals to `a[l]` because of this skip connection. What this means is that adding these two layers in the neural network doesn't really hurt the neural network's ability to do as well as this simpler network without these two extra layers, because it's quite easy for it to learn the identity function to just copy `a[l]` to `a[l+2]` despite the addition of these two layers. + - So adding two extra layers or adding this residual block to somewhere in the middle or the end of this big neural network doesn't hurt performance. It is easier to go from a decent baseline of not hurting performance and then gradient descent can only improve the solution from there. + +*About dimensions*: + +- In `a[l+2]=g(Z[l+2]+a[l])` we're assuming that `Z[l+2]` and `a[l]` have the same dimension. So what we see in ResNet is a lot of use of same convolutions. +- In case the input and output have different dimensions, we can add an extra matrix `W_s` so that `a[l+2] = g(Z[l+2] + W_s * a[l])`. The matrix `W_s` could be a matrix of parameters we learned or could be a fixed matrix that just implements zero paddings. + +*An example from the paper*: + +A plain network in which you input an image and then have a number of `CONV` layers until eventually you have a softmax output at the end. + +![resnet-plain-34](../_resources/resnet-plain-34.png) + +To turn this into a ResNet, you add those extra skip connections and there are a lot of `3x3` convolutions and most of these are `3x3` same convolutions and that's why you're adding equal dimension feature vectors. There are occasionally pooling layers and in these cases you need to make an adjustment to the dimension by the matrix `W_s`. + +![resnet-resnet-34](../_resources/resnet-resnet-34.png) + +**Practice advices on ResNet**: + +- Very deep "plain" networks don't work in practice because they are hard to train due to vanishing gradients. +- The skip-connections help to address the Vanishing Gradient problem. They also make it easy for a ResNet block to learn an identity function. +- There are two main types of blocks: The identity block and the convolutional block. +- Very deep Residual Networks are built by stacking these blocks together. + +#### Networks in Networks and 1x1 Convolutions + +Paper: [Network in Network](https://arxiv.org/abs/1312.4400) + +- At first, a 1×1 convolution does not seem to make much sense. After all, a convolution correlates adjacent pixels. A 1×1 convolution obviously does not. +- Because the minimum window is used, the 1×1 convolution loses the ability of larger convolutional layers to recognize patterns consisting of interactions among adjacent elements in the height and width dimensions. The only computation of the 1×1 convolution occurs on the channel dimension. +- The 1×1 convolutional layer is typically used to *adjust the number of channels* between network layers and to control model complexity. + +![conv-1x1](../_resources/conv-1x1.svg) + +*(image from [here](https://d2l.ai/chapter_convolutional-neural-networks/channels.html#times-1-convolutional-layer))* + +The 1×1 convolutional layer is equivalent to *the fully-connected layer*, when applied on a per pixel basis. + +- You can take every pixel as an *example* with `n_c[l]` input values (channels) and the output layer has `n_c[l+1]` nodes. The kernel is just nothing but the weights. +- Thus the 1x1 convolutional layer requires `n_c[l+1] x n_c[l]` weights and the bias. + +The 1x1 convolutional layer is actually doing something pretty non-trivial and adds non-linearity to your neural network and allow you to decrease or keep the same or if you want, increase the number of channels in your volumes. + +#### Inception Network Motivation + +Paper: [Going Deeper with Convolutions](https://arxiv.org/abs/1409.4842) + +When designing a layer for a ConvNet, you might have to pick, do you want a 1 by 3 filter, or 3 by 3, or 5 by 5, or do you want a pooling layer? +What the inception network does is it says, why shouldn't do them all? And this makes the network architecture more complicated, but it also works remarkably well. + +![inception-motivation](../_resources/inception-motivation.png) + +And the basic idea is that instead of you need to pick one of these filter sizes or pooling you want and commit to that, you can do them all and just concatenate all the outputs, and let the network learn whatever parameters it wants to use, whatever the combinations of these filter sizes it wants. Now it turns out that there is a problem with the inception layer as we've described it here, which is *computational cost*. + +*The analysis of computational cost*: + +![inception-computational-cost](../_resources/inception-computation.png) + +*Inception modules*: + +![inception](../_resources/inception.png) + +#### Inception Network + +![inception-module](../_resources/inception-module.png) + +- In order to really concatenate all of these outputs at the end we are going to use the same type of padding for pooling. +- What the inception network does is more or less put a lot of these modules together. + +![inception-network](../_resources/inception-network.png) + +The last few layers of the network is a fully connected layer followed by a softmax layer to try to make a prediction. What these side branches do is it takes some hidden layer and it tries to use that to make a prediction. You should think of this as maybe just another detail of the inception that's worked. But what is does is it helps to ensure that the features computed even in the heading units, even at intermediate layers that they're not too bad for protecting the output cause of a image. And this appears to have a regularizing effect on the inception network and helps prevent this network from overfitting. + +### Practical advices for using ConvNets + +#### Using Open-Source Implementation + +- Starting with open-source implementations is a better way, or certainly a faster way to get started on a new project. +- One of the advantages of doing so also is that sometimes these networks take a long time to train, and someone else might have used multiple GPUs and + a very large dataset to pretrain some of these networks. And that allows you to do transfer learning using these networks. + +#### Transfering Learning + +The computer vision research community has been pretty good at posting lots of data sets on the Internet so if you hear of things like ImageNet, or MS COCO, or PASCAL types of data sets, these are the names of different data sets that people have post online and a lot of computer researchers have trained their algorithms on. + +- [ImageNet](http://image-net.org/): ImageNet is an image database organized according to the WordNet hierarchy (currently only the nouns), in which each node of the hierarchy is depicted by hundreds and thousands of images. +- [Microsoft COCO](https://www.microsoft.com/en-us/research/publication/microsoft-coco-common-objects-in-context/): COCO is a common object in context. The dataset contains 91 objects types of 2.5 million labeled instances across 328,000 images. +- [PASCAL](https://www.cs.stanford.edu/~roozbeh/pascal-context/): PASCAL-Context Dataset This dataset is a set of additional annotations for PASCAL VOC 2010. It goes beyond the original PASCAL semantic segmentation task by providing annotations for the whole scene. The statistics section has a full list of 400+ labels. + +Sometimes these training takes several weeks and might take many GPUs and the fact that someone else has done this and gone through the painful high-performance search process, means that you can often download open source ways that took someone else many weeks or months to figure out and use that as a very good initialization for your own neural network. + +- If you have a small dataset for your image classification problem, you can download some open source implementation of a neural network and download not just the code but also the weights. And then you get rid of the softmax layer and create your own softmax unit that outputs your classification labels. +- To do this, you just freeze the parameters which you don't want to train. A lot of popular learning frameworks support this mode of operation (i.e., set *trainable* parameter to 0). +- Those early frozen layers are some fixed function that doesn't change. So one trick that could speedup training is that we just pre-compute that layer's activations and save them to disk. The advantage of the save-to-disk or the pre-compute method is that you don't need to recompute those activations everytime you take an epoch or take a path through a training set. +- If you have a larger label dataset one thing you could do is then freeze fewer layers. If you have a lot of data, in the extreme case, you could just use the downloaded weights as initialization so they would replace random initialization. + +#### Data Augmentation + +Having more data will help all computer vision tasks. + +*Some common data augmentation in computer vision*: + +- Mirroring +- Random cropping +- Rotation +- Shearing +- Local warping + +*Color shifting*: Take different values of R, G and B and use them to *distort the color channels*. In practice, the values R, G and B are drawn from some probability distribution. This makes your learning algorithm more robust to changes in the colors of your images. + +- One of the ways to implement color distortion uses an algorithm called PCA. The details of this are actually given in the AlexNet paper, and sometimes called PCA Color Augmentation. + - If your image is mainly purple, if it mainly has red and blue tints, and very little green, then PCA Color Augmentation, will add and subtract a lot to red and blue, where it balance \[inaudible\] all the greens, so kind of keeps the overall color of the tint the same. + +*Implementation tips*: + +A pretty common way of implementing data augmentation is to really have one thread, almost four threads, that is responsible for loading the data and implementing distortions, and then passing that to some other thread or some other process that then does the training. + +- Often the data augmentation and training process can run in parallel. +- Similar to other parts of training a deep neural network, the data augmentation process also has a few hyperparameters, such as how much color shifting do you implement and what parameters you use for random cropping. + +![data-augmentation-implementation](../_resources/data-augmentation.png) + +#### State of Computer Vision + +- Image recognition: the problem of looking at a picture and telling you is this a cat or not. +- Object detection: look in the picture and actually you're putting the bounding boxes are telling you where in the picture the objects, such as the car as well. The cost of getting the bounding boxes is more expensive to label the objects. + +*Data vs. hand-engineering*: + +- Having a lot of data: simpler algorithms as well as less hand-engineering. So less needing to carefully design features for the problem, but instead you can have a giant neural network, even a simpler architecture. +- Don't have much data: more hand-engineering ("hacks") + +*Two sources of knowledge*: + +- Labeled data, (x,y) +- Hand-engineering: features / network architecture / other components + +![data vs. hand-engineering](../_resources/data-hand-engineering.png) + +Even though data sets are getting bigger and bigger, often we just don't have as much data as we need. And this is why the computer vision historically and even today has relied more on hand-engineering. And this is also why that the field of computer vision has developed rather complex network architectures, is because in the absence of more data. The way to get good performance is to spend more time architecting, or fooling around with the network architecture. + +- Hand-engineering is very difficult and skillful task that requires a lot of insight. Historically the field of the computer vision has used very small datasets and the computer vision literature has relied on a lot of hand-engineering. +- In the last few years the amount of data with the computer vision task has increased so dramatically that the amount of hand-engineering has a significant reduction. +- But there's still a lot of hand-engineering of network architectures and computer vision, which is why you see very complicated hyperparameters choices in computer vision. +- The algorithms of object detection become even more complex and has even more specialized components. +- One thing that helps a lot when you have little data is *transfer learning*. + +**Tips for doing well on benchmarks/winning competitions**: + +- (1) Ensembling + - Train several networks independently and average their outputs (not weights). + - That maybe gives you 1% or 2% better, which really helps win a competition. + - To test on each image you might need to run an image through 3 to 15 different networks, so ensembling slows down your running time by a factor of 3 to 15. + - So ensembling is one of those tips that people use doing well in benchmarks and for winning competitions. + - Almost never use in production to serve actual customers. + - One big problem: need to keep all these different networks around, which takes up a lot more computer memory. +- (2) Multi-crop at test time + - Run classifier on multiple versions of test images and average results. + - Used much more for doing well on benchmarks than in actual production systems. + - Keep just one network around, which doesn't suck up as much memory, but it still slows down your run time quite a bit. + +![multi-crop](../_resources/multi-crop.png) + +*Use open source code*: + +- Use architectures of networks published in the literature +- Use open source implementations if possible +- Use pretrained models and fine-tune on your dataset + +#### Tips for Keras + +- Keras is a tool for rapid prototyping which allows you to quickly try out different model architectures. Only four steps to build a model using Keras: + - *Create*: define your model architecture, using functions such as `Input()`, `ZeroPadding2D()`, `Conv2D()`, `BatchNormalization()`, `MaxPooling2D()`, ... These python objects would be used as functions. [Know more about "Objects as functions"](https://medium.com/python-pandemonium/function-as-objects-in-python-d5215e6d1b0d). + - *Compile*: `model.compile(optimizer = "...", loss = "...", metrics = ["accuracy"])`. Optimizers include 'adam', 'sgd' or others. The loss function can be 'binary\_crossentropy' or 'categorical\_crossentropy' or others. See [Keras API Doc](https://keras.io/api/). + - *Fit/Train*: train the model by `model.fit(x = ..., y = ..., epochs = ..., batch_size = ...)`. + - *Evaluate/Test*: test the model by `model.evaluate(x = ..., y = ...)`. +- Model visualization tools: + - *Summarize model*: `model.summary()` prints the details of your layers in a table with the sizes of its inputs/outputs + - *Visualize model*: `plot_model()` plots your graph in a nice layout. + +For a full guidance read the newest tutorial on the Keras documentation: + +- [Introduction to Keras for Engineers](https://keras.io/getting_started/intro_to_keras_for_engineers/) +- [Introduction to Keras for Researchers](https://keras.io/getting_started/intro_to_keras_for_researchers/) + +Implementations of VGG16, ResNet and Inception by Keras can be found in [Francois Chollet's GitHub repository](https://github.com/fchollet/deep-learning-models). + +## Week 3: Object detection + +### Learning Objectives + +- Describe the challenges of Object Localization, Object Detection and Landmark Finding +- Implement non-max suppression to increase accuracy +- Implement intersection over union +- Label a dataset for an object detection application +- Identify the components used for object detection (landmark, anchor, bounding box, grid, ...) and their purpose + +### Detection algorithms + +#### Object Localization + +![object-classification-detection](../_resources/object-clf-detect.png) + +- The classification and the classification of localization problems usually have one object. +- In the detection problem there can be multiple objects. +- The ideas you learn about image classification will be useful for classification with localization, and the ideas you learn for localization will be useful for detection. + +![object-classification-localization](../_resources/object-clf-local.png) + +Giving the bounding box then you can use supervised learning to make your algorithm outputs not just a class label but also the four parameters to tell you where is the bounding box of the object you detected. + +![object-classification-localization-y](../_resources/object-clf-local-y.png) + +The squared error is used just to simplify the description here. In practice you could probably use a log like feature loss for the `c1, c2, c3` to the softmax output. + +#### Landmark Detection + +In more general cases, you can have a neural network just output x and y coordinates of important points in image, sometimes called landmarks. + +![landmark-detection](../_resources/object-landmark.png) + +If you are interested in people pose detection, you could also define a few key positions like the midpoint of the chest, the left shoulder, left elbow, the wrist, and so on. + +The identity of landmark one must be consistent across different images like maybe landmark one is always this corner of the eye, landmark two is always this corner of the eye, landmark three, landmark four, and so on. + +#### Object Detection + +![sliding windows detection](../_resources/object-slide-window.png) + +Disadvantage of sliding windows detection is computational cost. Unless you use a very fine granularity or a very small stride, you end up not able to localize the objects accurately within the image. + +#### Convolutional Implementation of Sliding Windows + +To build up towards the convolutional implementation of sliding windows let's first see how you can turn fully connected layers in neural network into convolutional layers. + +![Turn FC into CONV layers](../_resources/object-sliding-conv.png) + +What the convolutional implementation of sliding windows does is it allows *four* processes in the convnet to share a lot of computation. Instead of doing it sequentially, with the convolutional implementation you can implement the entire image, all maybe 28 by 28 and convolutionally make all the predictions at the same time. + +![convolutional implementation of sliding windows](../_resources/object-sliding-conv2.png) + +#### Bounding Box Predictions (YOLO) + +The convolutional implementation of sliding windows is more computationally efficient, but it still has a problem of not quite outputting the most accurate bounding boxes. The perfect bounding box isn't even quite square, it's actually has a slightly wider rectangle or slightly horizontal aspect ratio. + +![YOLO](../_resources/object-yolo-alg.png) + +**YOLO algorithm**: + +The basic idea is you're going to take the image classification and localization algorithm and apply that to each of the nine grid cells of the image. If the center/midpoint of an object falls into a grid cell, that grid cell is responsible for detecting that object. + +The advantage of this algorithm is that the neural network outputs precise bounding boxes as follows. + +- First, this allows in your network to output bounding boxes of any aspect ratio, as well as, output much more precise coordinates than are just dictated by the stride size of your sliding windows classifier. +- Second, this is a convolutional implementation and you're not implementing this algorithm nine times on the 3 by 3 grid or 361 times on 19 by 19 grid. + +#### Intersection Over Union + +`IoU` is a measure of the overlap between two bounding boxes. If we use `IoU` in the output assessment step, then the higher the `IoU` the more accurate the bounding box. However `IoU` is a nice tool for the YOLO algorithm to discard redundant bounding boxes. + +![IoU](../_resources/object-iou.png) + +#### Non-max Suppression + +One of the problems of Object Detection as you've learned about this so far, is that your algorithm may find multiple detections of the same objects. Rather than detecting an object just once, it might detect it multiple times. Non-max suppression is a way for you to make sure that your algorithm detects each object only once. + +- It first takes the largest `Pc` with the probability of a detection. +- Then, the non-max suppression part is to get rid of any other ones with a high (defined by a threshold) `IoU` between the box chosen in the first step. + +![Non-max](../_resources/object-nonmax.png) + +If you actually tried to detect three objects say pedestrians, cars, and motorcycles, then the output vector will have three additional components. And it turns out, the right thing to do is to independently carry out non-max suppression three times, one on each of the outputs classes. + +#### Anchor Boxes + +One of the problems with object detection as you have seen it so far is that each of the grid cells can detect only one object. What if a grid cell wants to detect multiple objects? This is what the idea of anchor boxes does. + +*Anchor box algorithm*: + +| previous box | with two anchor boxes | +| --- | --- | +| Each object in training image is assigned to grid cell that contains that object’s midpoint. | Each object in training image is assigned to grid cell that contains object’s midpoint and anchor box for the grid cell with highest `IoU`. | +| Output `y`: `3x3x8` | Output `y`: `3x3x16` or `3x3x2x8` | + +![anchor box](../_resources/object-anchorbox.png) + +#### YOLO Algorithm + +*YOLO algorithm steps*: + +- If you're using two anchor boxes, then for each of the nine grid cells, you get two predicted bounding boxes. +- Next, you then get rid of the low probability predictions. +- And then finally if you have three classes you're trying to detect, you're trying to detect pedestrians, cars and motorcycles. What you do is, for each of the three classes, independently run non-max suppression for the objects that were predicted to come from that class. + +![yolo-algorithm](../_resources/object-yolo-algorithm.png) + +#### (Optional) Region Proposals + +| algorithm | description | +| --- | --- | +| R-CNN | Propose regions. Classify proposed regions one at a time. Output label + bounding box. The way that they perform the region proposals is to run an algorithm called a segmentation algorithm. One downside of the R-CNN algorithm was that it is actually quite slow. | +| Fast R-CNN | Propose regions. Use convolution implementation of sliding windows to classify all the proposed regions. One of the problems of fast R-CNN algorithm is that the clustering step to propose the regions is still quite slow. | +| Faster R-CNN | Use convolutional network to propose regions. (Most implementations are usually still quit a bit slower than the YOLO algorithm.) | + +## Week 4: Special applications: Face recognition & Neural style transfer + +Discover how CNNs can be applied to multiple fields, including art generation and face recognition. Implement your own algorithm to generate art and recognize faces. + +### Face Recognition + +#### What is face recognition + +- Verification + - Input image, name/ID + - Output whether the input image is that of the claimed person +- Recognition + - Has a database of K persons + - Get an input image + - Output ID if the image is any of the K persons (or “not recognized”) + +#### One Shot Learning + +One-shot learning problem: to recognize a person given just one single image. + +- So one approach is to input the image of the person, feed it too a ConvNet. And have it output a label, y, using a softmax unit with four outputs or maybe five outputs corresponding to each of these four persons or none of the above. However, this doesn't work well. +- Instead, to make this work, what you're going to do instead is learn a **similarity function** `d(img1,img2) = degree of difference between images`. So long as you can learn this function, which inputs a pair of images and tells you, basically, if they're the same person or different persons. Then if you have someone new join your team, you can add a fifth person to your database, and it just works fine. + +#### Siamese network + +A good way to implement a *similarity function* `d(img1, img2)` is to use a [Siamese network](https://www.paperswithcode.com/method/siamese-network). + +![siamese-network](../_resources/siamese-network.png) + +In a Siamese network, instead of making a classification by a softmax unit, we focus on the vector computed by a fully connected layer as an encoding of the input image `x1`. + +*Goal of learning*: + +- Parameters of NN define an encoding `𝑓(𝑥_𝑖)` +- Learn parameters so that: + - If `𝑥_𝑖,𝑥_𝑗` are the same person, `‖f(𝑥_𝑖)−f(𝑥_𝑗)‖^2` is small. + - If `𝑥_𝑖,𝑥_𝑗` are different persons, `‖f(𝑥_𝑖)−f(𝑥_𝑗)‖^2` is large. + +#### Triplet Loss + +One way to learn the parameters of the neural network so that it gives you a good encoding for your pictures of faces is to define an applied gradient descent on the triplet loss function. + +In the terminology of the triplet loss, what you're going do is always look at one anchor image and then you want to distance between the anchor and the positive image, really a positive example, meaning as the same person to be similar. Whereas, you want the anchor when pairs are compared to the negative example for their distances to be much further apart. You'll always be looking at three images at a time: + +- an anchor image (A) +- a positive image (P) +- a negative image (N) + +As before we have `d(A,P)=‖f(A)−f(P)‖^2` and `d(A,N)=‖f(A)−f(N)‖^2`, the learning objective is to have `d(A,P) ≤ d(A,N)`. But if `f` always equals zero or `f` always outputs the same, i.e., the encoding for every image is identical, the objective is easily achieved, which is not what we want. So we need to add an `𝛼` to the left, a margin, which is a terminology you can see on support vector machines. + +*The learning objective*: + +`d(A,P) + 𝛼 ≤ d(A,N)` or `d(A,P) - d(A,N) + 𝛼 ≤ 0` + +*Loss function*: + +``` +Given 3 images A,P,N: +L(A,P,N) = max(d(A,P) - d(A,N) + 𝛼, 0) +J = sum(L(A[i],P[i],N[i])) +``` + +You do need a dataset where you have multiple pictures of the same person. If you had just one picture of each person, then you can't actually train this system. + +- During training, if A,P,N are chosen randomly, `𝑑(𝐴,𝑃) + 𝛼 ≤ 𝑑(𝐴,𝑁)` is easily satisfied. +- Choose triplets that're "hard" to train on. + +#### Face Verification and Binary Classification + +The Triplet loss is a good way to learn the parameters of a ConvNet for face recognition. Face recognition can also be posed as a straight binary classification problem by taking a pair of neural networks to take a Siamese Network and having them both compute the embeddings, maybe 128 dimensional embeddings or even higher dimensional, and then having the embeddings be input to a logistic regression unit to make a prediction. The output will be one if both of them are the same person and zero if different. + +![face-recognition](../_resources/face-recognition.png) + +*Implementation tips*: + +Instead of having to compute the encoding every single time you can pre-compute that, which can save a significant computation. + +#### Summary of Face Recognition + +*Key points to remember*: + +- Face verification solves an easier 1:1 matching problem; face recognition addresses a harder 1:K matching problem. +- The triplet loss is an effective loss function for training a neural network to learn an encoding of a face image. +- The same encoding can be used for verification and recognition. Measuring distances between two images' encodings allows you to determine whether they are pictures of the same person. + +*More references*: + +- Florian Schroff, Dmitry Kalenichenko, James Philbin (2015). [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://arxiv.org/pdf/1503.03832.pdf) +- Yaniv Taigman, Ming Yang, Marc'Aurelio Ranzato, Lior Wolf (2014). [DeepFace: Closing the gap to human-level performance in face verification](https://research.fb.com/wp-content/uploads/2016/11/deepface-closing-the-gap-to-human-level-performance-in-face-verification.pdf) +- The pretrained model we use is inspired by Victor Sy Wang's implementation and was loaded using his code: https://github.com/iwantooxxoox/Keras-OpenFace. +- Our implementation also took a lot of inspiration from the official FaceNet github repository: https://github.com/davidsandberg/facenet + +### Neural Style Transfer + +#### What is neural style transfer + +Paper: [A Neural Algorithm of Artistic Style](https://arxiv.org/abs/1508.06576) + +![neural style transfer](../_resources/neural-style-transfer.png) + +In order to implement Neural Style Transfer, you need to look at the features extracted by ConvNet at various layers, the shallow and the deeper layers of a ConvNet. + +#### What are deep ConvNets learning + +Paper: [Visualizing and Understanding Convolutional Networks](https://arxiv.org/abs/1311.2901) + +![visualizing network](../_resources/visualizing-nn.png) + +#### Cost Function + +*Neural style transfer cost function*: + +``` +J(G) = alpha * J_content(C, G) + beta * J_style(S, G) +``` + +*Find the generated image G*: + +1. Initiate G randomly, `G: 100 x 100 x 3` +2. Use gradient descent to minimize `J(G)` + +#### Content Cost Function + +- Say you use hidden layer 𝑙 to compute content cost. (Usually, choose some layer in the middle, neither too shallow nor too deep) +- Use pre-trained ConvNet. (E.g., VGG network) +- Let `𝑎[𝑙](𝐶)` and `𝑎[𝑙](𝐺)` be the activation of layer 𝑙 on the images +- If `𝑎[𝑙](𝐶)` and `𝑎[𝑙](𝐺)` are similar, both images have similar content + +``` +J_content(C, G) = 1/2 * ‖𝑎[𝑙](𝐶)−𝑎[𝑙](𝐺)‖^2 +``` + +#### Style Cost Function + +Style is defined as correlation between activations across channels. + +![style-cost1](../_resources/style-cost1.png) + +![style-cost2](../_resources/style-cost2.png) + +![style-cost3](../_resources/style-cost3.png) + +#### 1D and 3D Generalizations + +ConvNets can apply not just to 2D images but also to 1D data as well as to 3D data. + +For 1D data, like ECG signal (electrocardiogram), it's a time series showing the voltage at each instant time. Maybe we have a 14 dimensional input. With 1D data applications, we actually use a recurrent neural network. + +``` +14 x 1 * 5 x 1 --> 10 x 16 (16 filters) +``` + +For 3D data, we can think the data has some height, some width, and then also some depth. For example, we want to apply a ConvNet to detect features in a 3D CT scan, for simplifying purpose, we have 14 x 14 x 14 input here. + +``` +14 x 14 x 14 x 1 * 5 x 5 x 5 x 1 --> 10 x 10 x 10 x 16 (16 filters) +``` + +Other 3D data can be movie data where the different slices could be different slices in time through a movie. We could use ConvNets to detect motion or people taking actions in movies. + +* * * + +Notes by Aaron © 2020 \ No newline at end of file diff --git a/DeepLearning_AI/Course 5_ Sequence Models.md b/DeepLearning_AI/Course 5_ Sequence Models.md new file mode 100644 index 0000000..f827d45 --- /dev/null +++ b/DeepLearning_AI/Course 5_ Sequence Models.md @@ -0,0 +1,935 @@ +--- +title: 'Course 5: Sequence Models' +updated: 2022-05-17 19:02:05Z +created: 2022-05-16 17:55:13Z +--- + +# Course 5: Sequence Models + +- [Course 5: Sequence Models](#course-5-sequence-models) + - [Week 1: Recurrent Neural Networks](#week-1-recurrent-neural-networks) + - [Recurrent Neural Networks](#recurrent-neural-networks) + - [Why sequence models](#why-sequence-models) + - [Notation](#notation) + - [Recurrent Neural Network Model](#recurrent-neural-network-model) + - [Backpropagation through time](#backpropagation-through-time) + - [Different types of RNNs](#different-types-of-rnns) + - [Language model and sequence generation](#language-model-and-sequence-generation) + - [Sampling novel sequences](#sampling-novel-sequences) + - [Vanishing gradients with RNNs](#vanishing-gradients-with-rnns) + - [Gated Recurrent Unit (GRU)](#gated-recurrent-unit-gru) + - [Long Short Term Memory (LSTM)](#long-short-term-memory-lstm) + - [Bidirectional RNN](#bidirectional-rnn) + - [Deep RNNs](#deep-rnns) + - [Week 2: Natural Language Processing & Word Embeddings](#week-2-natural-language-processing--word-embeddings) + - [Introduction to Word Embeddings](#introduction-to-word-embeddings) + - [Word Representation](#word-representation) + - [Using word embeddings](#using-word-embeddings) + - [Properties of word embeddings](#properties-of-word-embeddings) + - [Embedding matrix](#embedding-matrix) + - [Learning Word Embeddings: Word2vec & GloVe](#learning-word-embeddings-word2vec--glove) + - [Learning word embeddings](#learning-word-embeddings) + - [Word2Vec](#word2vec) + - [Negative Sampling](#negative-sampling) + - [GloVe word vectors](#glove-word-vectors) + - [Applications using Word Embeddings](#applications-using-word-embeddings) + - [Sentiment Classification](#sentiment-classification) + - [Debiasing word embeddings](#debiasing-word-embeddings) + - [Week 3: Sequence models & Attention mechanism](#week-3-sequence-models--attention-mechanism) + - [Various sequence to sequence architectures](#various-sequence-to-sequence-architectures) + - [Basic Models](#basic-models) + - [Picking the most likely sentence](#picking-the-most-likely-sentence) + - [Beam Search](#beam-search) + - [Refinements to Beam Search](#refinements-to-beam-search) + - [Error analysis in beam search](#error-analysis-in-beam-search) + - [Bleu Score (optional)](#bleu-score-optional) + - [Attention Model Intuition](#attention-model-intuition) + - [Attention Model](#attention-model) + - [Speech recognition - Audio data](#speech-recognition---audio-data) + - [Speech recognition](#speech-recognition) + - [Trigger Word Detection](#trigger-word-detection) + +## Week 1: Recurrent Neural Networks + +> Learn about recurrent neural networks. This type of model has been proven to perform extremely well on temporal data. It has several variants including LSTMs, GRUs and Bidirectional RNNs, which you are going to learn about in this section. + +### Recurrent Neural Networks + +#### Why sequence models + +Examples of sequence data: + +- Speech recognition +- Music generation +- Sentiment classification +- DNA sequence analysis +- Machine translation +- Video activity recognition +- Named entity recognition + +#### Notation + +For a motivation, in the problem of Named Entity Recognition (NER), we have the following notation: + +- `x` is the input sentence, such as: `Harry Potter and Hermione Granger invented a new spell.` +- `y` is the output, in this case: `1 1 0 1 1 0 0 0 0`. +- x<t> denote the word in the index `t` and y<t> is the correspondent output. +- In the *i*-th input example, x(i)<t> is *t*-th word and Tx(i) is the length of the *i*-th example. +- Ty is the length of the output. In NER, we have Tx = Ty. + +Words representation introduced in this video is the One-Hot representation. + +- First, you have a dictionary which words appear in a certain order. +- Second, for a particular word, we create a new vector with `1` in position of the word in the dictionary and `0` everywhere else. + +For a word not in your vocabulary, we need create a new token or a new fake word called unknown word denoted by ``. + +#### Recurrent Neural Network Model + +If we build a neural network to learn the mapping from x to y using the one-hot representation for each word as input, it might not work well. There are two main problems: + +- Inputs and outputs can be different lengths in different examples. not every example has the same input length Tx or the same output length Ty. Even with a maximum length, zero-padding every input up to the maximum length doesn't seem like a good representation. +- For a naive neural network architecture, it doesn't share features learned across different positions of texts. + +*Recurrent Neural Networks*: + +- A recurrent neural network does not have either of these disadvantages. +- At each time step, the recurrent neural network that passes on as activation to the next time step for it to use. +- The recurrent neural network scans through the data from left to right. The parameters it uses for each time step are shared. +- One limitation of unidirectional neural network architecture is that the prediction at a certain time uses inputs or uses information from the inputs earlier in the sequence but not information later in the sequence. + - `He said, "Teddy Roosevelt was a great president."` + - `He said, "Teddy bears are on sale!"` + - You can't tell the difference if you look only at the first three words. + +![rnn-forward](../_resources/rnn-forward.png) + +Instead of carrying around two parameter matrices Waa and Wax, we can simplifying the notation by compressing them into just one parameter matrix Wa. + +![rnn-notation](../_resources/rnn-notation.png) + +#### Backpropagation through time + +In the backpropagation procedure the most significant messaage or the most significant recursive calculation is which goes from right to left, that is, backpropagation through time. + +#### Different types of RNNs + +There are different types of RNN: + +- One to One +- One to Many +- Many to One +- Many to Many + +![rnn-type](../_resources/rnn-type.png) + +See more details about RNN by [Karpathy](http://karpathy.github.io/2015/05/21/rnn-effectiveness/). + +#### Language model and sequence generation + +So what a language model does is to tell you what is the probability of a particular sentence. + +For example, we have two sentences from speech recognition application: + +| sentence | probability | +| --- | --- | +| The apple and pair salad. | 𝑃(The apple and pair salad)=3.2x10-13 | +| The apple and pear salad. | 𝑃(The apple and pear salad)=5.7x10-10 | + +For language model it will be useful to represent a sentence as output `y` rather than inputs `x`. So what the language model does is to estimate the probability of a particular sequence of words `𝑃(y<1>, y<2>, ..., y)`. + +*How to build a language model*? + +`Cats average 15 hours of sleep a day ` Totally 9 words in this sentence. + +- The first thing you would do is to tokenize this sentence. +- Map each of these words to one-hot vectors or indices in vocabulary. + - Maybe need to add extra token for end of sentence as `` or unknown words as ``. + - Omit the period. if you want to treat the period or other punctuation as explicit token, then you can add the period to you vocabulary as well. +- Set the inputs x<t> = y<t-1>. +- What `a1` does is it will make a softmax prediction to try to figure out what is the probability of the first words y<1>. That is what is the probability of any word in the dictionary. Such as, what's the chance that the first word is *Aaron*? +- Until the end, it will predict the chance of ``. +- Define the cost function. The overall loss is just the sum over all time steps of the loss associated with the individual predictions. + +![language model](../_resources/rnn-lm.png) + +If you train this RNN on a large training set, we can do: + +- Given an initial set of words, use the model to predict the chance of the next word. +- Given a new sentence `y<1>,y<2>,y<3>`, use it to figure out the chance of this sentence: `p(y<1>,y<2>,y<3>) = p(y<1>) * p(y<2>|y<1>) * p(y<3>|y<1>,y<2>)` + +#### Sampling novel sequences + +After you train a sequence model, one way you can informally get a sense of what is learned is to have it sample novel sequences. + +*How to generate a randomly chosen sentence from your RNN language model*: + +- In the first time step, sample what is the first word you want your model to generate: randomly sample according to the softmax distribution. + - What the softmax distribution gives you is it tells the chance of the first word is 'a', the chance of the first word is 'Aaron', the chance of the first word is 'Zulu', or the chance of the first word refers to `` or ``. All these probabilities can form a vector. + - Take the vector and use `np.random.choice` to sample according to distribution defined by this vector probabilities. That lets you sample the first word. +- In the second time step, remember in the last section, y<1> is expected as input. Here take ŷ<1> you just sampled and pass it as input to the second step. Then use `np.random.choice` to sample ŷ<2>. Repeat this process until you generate an `` token. +- If you want to make sure that your algorithm never generate ``, just reject any sample that come out as `` and keep resampling from vocabulary until you get a word that's not ``. + +*Character level language model*: + +If you build a character level language model rather than a word level language model, then your sequence y1, y2, y3, would be the individual characters in your training data, rather than the individual words in your training data. Using a character level language model has some pros and cons. As computers gets faster there are more and more applications where people are, at least in some special cases, starting to look at more character level models. + +- Advantages: + - You don't have to worry about ``. +- Disadvantages: + - The main disadvantage of the character level language model is that you end up with much longer sequences. + - And so character language models are not as good as word level language models at capturing long range dependencies between how the the earlier parts of the sentence also affect the later part of the sentence. + - More computationally expensive to train. + +#### Vanishing gradients with RNNs + +- One of the problems with a basic RNN algorithm is that it runs into vanishing gradient problems. +- Language can have very long-term dependencies, for example: + - The **cat**, which already ate a bunch of food that was delicious ..., **was** full. + - The **cats**, which already ate a bunch of food that was delicious, and apples, and pears, ..., **were** full. +- The basic RNN we've seen so far is not very good at capturing very long-term dependencies. It's difficult for the output to be strongly influenced by an input that was very early in the sequence. +- When doing backprop, the gradients should not just decrease exponentially, they may also increase exponentially with the number of layers going through. +- Exploding gradients are easier to spot because the parameters just blow up and you might often see NaNs, or not a numbers, meaning results of a numerical overflow in your neural network computation. + - One solution to that is apply *gradient clipping*: it is bigger than some threshold, re-scale some of your gradient vector so that is not too big. +- Vanishing gradients is much harder to solve and it will be the subject of GRU or LSTM. + +#### Gated Recurrent Unit (GRU) + +Gate Recurrent Unit is one of the ideas that has enabled RNN to become much better at capturing very long range dependencies and has made RNN much more effective. + +A visualization of the RNN unit of the hidden layer of the RNN in terms of a picture: + +![rnn-unit](../_resources/rnn-unit.png) + +- The GRU unit is going to have a new variable called `c`, which stands for memory cell. +- c̃<t> is a candidate for replacing c<t>. +- For intuition, think of Γu as being either zero or one most of the time. In practice gamma won't be exactly zero or one. +- Because Γu can be so close to zero, can be 0.000001 or even smaller than that, it doesn't suffer from much of a vanishing gradient problem +- Because when Γu is so close to zero this becomes essentially c<t> = c<t-1> and the value of c is maintained pretty much exactly even across many many time-steps. So this can help significantly with the vanishing gradient problem and therefore allow a neural network to go on even very long range dependencies. +- In the full version of GRU, there is another gate Γr. You can think of `r` as standing for relevance. So this gate Γr tells you how relevant is c<t-1> to computing the next candidate for c<t>. + +![GRU](../_resources/GRU.png) + +*Implementation tips*: + +- The asterisks are actually element-wise multiplication. +- If you have 100 dimensional or hidden activation value, then c<t>, c̃<t>, Γu would be the same dimension. + - If Γu is 100 dimensional vector, then it is really a 100 dimensional vector of bits, the value is mostly zero and one. + - That tells you of this 100 dimensional memory cell which are the bits you want to update. What these element-wise multiplications do is it just element-wise tells the GRU unit which bits to update at every time-step. So you can choose to keep some bits constant while updating other bits. + - In practice gamma won't be exactly zero or one. + +#### Long Short Term Memory (LSTM) + +Fancy explanation: [Understanding LSTM Network](http://colah.github.io/posts/2015-08-Understanding-LSTMs/) + +- For the LSTM we will no longer have the case that a<t> is equal to c<t>. +- And we're not using relevance gate Γr. Instead, LSTM has update, forget and output gates, Γu, Γf and Γo respectively. + +![LSTM-units](../_resources/LSTM-units.png) + +One cool thing about this you'll notice is that this red line at the top that shows how, so long as you set the forget and the update gate appropriately, it is relatively easy for the LSTM to have some value c<0> and have that be passed all the way to the right to have your, maybe, c<3> equals c<0>. And this is why the LSTM, as well as the GRU, is very good at memorizing certain values even for a long time, for certain real values stored in the memory cell even for many, many timesteps. + +![LSTM](../_resources/LSTM.png) + +*One common variation of LSTM*: + +- Peephole connection: instead of just having the gate values be dependent only on a<t-1>, x<t>, sometimes, people also sneak in there the values c<t-1> as well. + +*GRU vs. LSTM*: + +- The advantage of the GRU is that it's a simpler model and so it is actually easier to build a much bigger network, it only has two gates, + so computationally, it runs a bit faster. So, it scales the building somewhat bigger models. +- The LSTM is more powerful and more effective since it has three gates instead of two. If you want to pick one to use, LSTM has been the historically more proven choice. Most people today will still use the LSTM as the default first thing to try. + +**Implementation tips**: + +- *forget gate Γf* + + - The forget gate Γf<t> has the same dimensions as the previous cell state c<t-1>. + - This means that the two can be multiplied together, element-wise. + - Multiplying the tensors Γf<t> is like applying a mask over the previous cell state. + - If a single value in Γf<t> is 0 or close to 0, then the product is close to 0. + - This keeps the information stored in the corresponding unit in c<t-1> from being remembered for the next time step. + - Similarly, if one value is close to 1, the product is close to the original value in the previous cell state. + - The LSTM will keep the information from the corresponding unit of c<t-1>, to be used in the next time step. +- *candidate value c̃<t>* + + - The candidate value is a tensor containing information from the current time step that **may** be stored in the current cell state c<t>. + - Which parts of the candidate value get passed on depends on the update gate. + - The candidate value is a tensor containing values that range from -1 to 1. (tanh function) + - The tilde "~" is used to differentiate the candidate c̃<t> from the cell state c<t>. +- *update gate Γu* + + - The update gate decides what parts of a "candidate" tensor c̃<t> are passed onto the cell state c<t>. + - The update gate is a tensor containing values between 0 and 1. + - When a unit in the update gate is close to 1, it allows the value of the candidate c̃<t> to be passed onto the hidden state c<t>. + - When a unit in the update gate is close to 0, it prevents the corresponding value in the candidate from being passed onto the hidden state. +- *cell state c<t>* + + - The cell state is the "memory" that gets passed onto future time steps. + - The new cell state c<t> is a combination of the previous cell state and the candidate value. +- *output gate Γo* + + - The output gate decides what gets sent as the prediction (output) of the time step. + - The output gate is like the other gates. It contains values that range from 0 to 1. +- *hidden state a<t>* + + - The hidden state gets passed to the LSTM cell's next time step. + - It is used to determine the three gates (Γf, Γu, Γo) of the next time step. + - The hidden state is also used for the prediction y<t>. + +#### Bidirectional RNN + +![RNN-ner](../_resources/BRNN-ner.png) + +- Bidirectional RNN lets you at a point in time to take information from both earlier and later in the sequence. +- This network defines a Acyclic graph +- The forward prop has part of the computation going from left to right and part of computation going from right to left in this diagram. +- So information from x<1>, x<2>, x<3> are all taken into account with information from x<4> can flow through a backward four to a backward three to Y three. So this allows the prediction at time three to take as input both information from the past, as well as information from the present which goes into both the forward and the backward things at this step, as well as information from the future. +- Blocks can be not just the standard RNN block but they can also be GRU blocks or LSTM blocks. In fact, BRNN with LSTM units is commonly used in NLP problems. + +![BRNN](../_resources/BRNN.png) + +*Disadvantage*: + +The disadvantage of the bidirectional RNN is that you do need the entire sequence of data before you can make predictions anywhere. So, for example, if you're building a speech recognition system, then the BRNN will let you take into account the entire speech utterance but if you use this straightforward implementation, you need to wait for the person to stop talking to get the entire utterance before you can actually process it and make a speech recognition prediction. For a real type speech recognition applications, they're somewhat more complex modules as well rather than just using the standard bidirectional RNN as you've seen here. + +#### Deep RNNs + +- For learning very complex functions sometimes is useful to stack multiple layers of RNNs together to build even deeper versions of these models. +- The blocks don't just have to be standard RNN, the simple RNN model. They can also be GRU blocks LSTM blocks. +- And you can also build deep versions of the bidirectional RNN. + +![DRNN](../_resources/DRNN.png) + +## Week 2: Natural Language Processing & Word Embeddings + +> Natural language processing with deep learning is an important combination. Using word vector representations and embedding layers you can train recurrent neural networks with outstanding performances in a wide variety of industries. Examples of applications are sentiment analysis, named entity recognition and machine translation. + +### Introduction to Word Embeddings + +#### Word Representation + +- One of the weaknesses of one-hot representation is that it treats each word as a thing unto itself, and it doesn't allow an algorithm to easily generalize across words. + + - Because the any product between any two different one-hot vector is zero. + - It doesn't know that somehow apple and orange are much more similar than king and orange or queen and orange. +- Instead we can learn a featurized representation. + + - But by a lot of the features of apple and orange are actually the same, or take on very similar values. And so, this increases the odds of the learning algorithm that has figured out that orange juice is a thing, to also quickly figure out that apple juice is a thing. + - The features we'll end up learning, won't have a easy to interpret interpretation like that component one is gender, component two is royal, component three is age and so on. What they're representing will be a bit harder to figure out. + - But nonetheless, the featurized representations we will learn, will allow an algorithm to quickly figure out that apple and orange are more similar than say, king and orange or queen and orange. + + | features\\words | Man (5391) | Woman (9853) | King (4914) | Queen (7157) | Apple (456) | Orange (6257) | + | --- | --- | --- | --- | --- | --- | --- | + | Gender | -1 | 1 | -0.95 | 0.97 | 0.00 | 0.01 | + | Royal | 0.01 | 0.02 | 0.93 | 0.95 | -0.01 | 0.00 | + | Age (adult?) | 0.03 | 0.02 | 0.7 | 0.69 | 0.03 | -0.02 | + | Food | 0.09 | 0.01 | 0.02 | 0.01 | 0.95 | 0.97 | + | Size | ... | ... | ... | ... | ... | ... | + | ... | ... | ... | ... | ... | ... | ... | + +- One common algorithm for visualizing word representation is the [t-SNE](http://www.cs.toronto.edu/~hinton/absps/tsne.pdf) algorithm due to [Laurens van der Maaten](https://lvdmaaten.github.io/tsne/) and Geoff Hinton. + + +#### Using word embeddings + +- Learn word embeddings from large text corpus. (1-100B words) (Or download pre-trained embedding online.) +- Transfer embedding to new task with smaller training set. (say, 100k words) +- Optional: Continue to finetune the word embeddings with new data. + - In practice, you would do this only if this task 2 has a pretty big data set. + - If your label data set for step 2 is quite small, then usually, I would not bother to continue to fine tune the word embeddings. + +Word embeddings tend to make the biggest difference when the task you're trying to carry out has a relatively smaller training set. + +- Useful for NLP standard tasks. + - Named entity recognition + - Text summarization + - Co-reference + - Parsing +- Less useful for: + - Language modeling + - Machine translation + +*Word embedding vs. face recognition encoding*: + +- The words encoding and embedding mean fairly similar things. In the face recognition literature, people also use the term encoding to refer to the vectors, `f(x(i))` and `f(x(j))`. Refer to [Course 4](joplin://1c1155ef678f4b41a1b0aa6fd36eabad#face-verification-and-binary-classification). +- For face recognition, you wanted to train a neural network that can take any face picture as input, even a picture you've never seen before, and have a neural network compute an encoding for that new picture. +- What we'll do for learning word embeddings is that we'll have a fixed vocabulary of, say, 10,000 words. We'll learn a fixed encoding or learn a fixed embedding for each of the words in our vocabulary. +- The terms encoding and embedding are used somewhat interchangeably. So the difference is not represented by the difference in terminologies. It's just a difference in how we need to use these algorithms in face recognition with unlimited pictures and natural language processing with a fixed vocabulary. + +#### Properties of word embeddings + +- Word embeddings can be used for analogy reasoning, which can help convey a sense of what word embeddings are doing even though analogy reasoning is not by itself the most important NLP application. + +- `man --> woman` vs. `king --> queen`: + eman \- ewoman ≈ eking \- equeen + +- To carry out an analogy reasoning, man is to woman as king is to what? + + - To find a word so that eman \- ewoman ≈ eking \- e?. + - Find word `w`: argmaxw *sim*(ew, eking-eman+ewoman) + - We can use cosine similarity to calculate this similarity. + - Refer to work paper by [Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/rvecs.pdf). +- What t-SNE does is, it takes 300-D data, and it maps it in a very non-linear way to a 2D space. And so the mapping that t-SNE learns, this is a very complicated and very non-linear mapping. So after the t-SNE mapping, you should not expect these types of parallelogram relationships, like the one we saw on the left, to hold true. And many of the parallelogram analogy relationships will be broken by t-SNE. + + ![word-embedding](../_resources/word-embedding.png) + + +#### Embedding matrix + +- When you implement an algorithm to learn a word embedding, what you end up learning is an embedding matrix. + + - E: embedding matrix (300, 10000) + + - O6257 = \[0,......0,1,0,...,0\], (10000, 1) + + - E·O6257 = e6257, (300, 1) + + | a | aaron | ... | orange (6257) | ... | zulu | `` | + | --- | --- | --- | --- | --- | --- | --- | + | ... | ... | ... | ... | ... | ... | ... | + | ... | ... | ... | ... | ... | ... | ... | + | ... | ... | ... | ... | ... | ... | ... | + +- Our goal will be to learn an embedding matrix E by initializing E randomly and then learning all the parameters of this (300, 10000) dimensional matrix. + +- E times the one-hot vector gives you the embedding vector. + +- In practice, use specialized function to look up an embedding. + + +### Learning Word Embeddings: Word2vec & GloVe + +#### Learning word embeddings + +- In the history of deep learning as applied to learning word embeddings, people actually started off with relatively complex algorithms. And then over time, researchers discovered they can use simpler and simpler and simpler algorithms and still get very good results especially for a large dataset. + +- A more complex algorithm: a neural language model, by Yoshua Bengio, Rejean Ducharme, Pascals Vincent, and Christian Jauvin: [A Neural Probabilistic Language Model](http://www.iro.umontreal.ca/~lisa/pointeurs/BengioDucharmeVincentJauvin_jmlr.pdf). + + - Let's start to build a neural network to predict the next word in the sequence below. + + ``` + I want a glass of orange ______. + 4343 9665 1 3852 6163 6257 + ``` + + ![n-lm](../_resources/neural-lm-embedding.png) + + - If we have a fixed historical window of 4 words (4 is a hyperparameter), then we take the four embedding vectors and stack them together, and feed them into a neural network, and then feed this neural network output to a softmax, and the softmax classifies among the 10,000 possible outputs in the vocab for the final word we're trying to predict. These two layers have their own parameters W1,b1 and W2, b2. + + - This is one of the earlier and pretty successful algorithms for learning word embeddings. + +- A more generalized algorithm. + + - We have a longer sentence: `I want a glass of orange juice to go along with my cereal`. The task is to predict the word `juice` in the middle. + - If it goes to build a language model then is natural for the context to be a few words right before the target word. But if your goal isn't to learn the language model per se, then you can choose other contexts. + - Contexts: + - Last 4 words: descibed previously. + - 4 words on left & right: `a glass of orange ___ to go along with` + - Last 1 word: `orange`, much more simpler context. + - Nearby 1 word: `glass`. This is the idea of a **Skip-Gram** model, which works surprisingly well. + - If your main goal is really to learn a word embedding, then you can use all of these other contexts and they will result in very meaningful work embeddings as well. + +#### Word2Vec + +Paper: [Efficient Estimation of Word Representations in Vector Space](https://arxiv.org/abs/1301.3781) by Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean. + +*The Skip-Gram model*: + +- In the skip-gram model, what we're going to do is come up with a few context to target errors to create our supervised learning problem. + +- So rather than having the context be always the last four words or the last end words immediately before the target word, what I'm going to do is, say, randomly pick a word to be the context word. And let's say we chose the word `orange`. + +- What we're going to do is randomly pick another word within some window. Say plus minus five words or plus minus ten words of the context word and we choose that to be target word. + + - maybe by chance pick `juice` to be a target word, that just one word later. + + - maybe `glass`, two words before. + + - maybe `my`. + + | Context | Target | + | --- | --- | + | orange | juice | + | orange | glass | + | orange | my | + +- And so we'll set up a supervised learning problem where given the context word, you're asked to predict what is a randomly chosen word within say, a ±10 word window, or a ±5 word window of the input context word. + +- This is not a very easy learning problem, because within ±10 words of the word `orange`, it could be a lot of different words. + +- But the goal of setting up this supervised learning problem isn't to do well on the supervised learning problem per se. It is that we want to use this learning problem to learning good word embeddings. + + +*Model details*: + +- Context `c`: `orange` and target `t`: `juice`. +- oc ---\> E ---> ec ---\> O(softmax) ---> ŷ. This is the little neural network with basically looking up the embedding and then just a softmax unit. +- Softmax: ![p(t|c)](../_resources/softmax.svg), 𝜃t: parameter associated with output `t`. (bias term is omitted) +- Loss: L(ŷ,y) = -sum(yilogŷi) +- So this is called the skip-gram model because it's taking as input one word like `orange` and then trying to predict some words skipping a few words from the left or the right side. + +*Model problem*: + +- Computational speed: in the softmax step, every time evaluating the probability, you need to carry out a sum over all 10,000, maybe even larger 1,000,000, words in your vocabulary. It gets really slow to do that every time. + +*Hierarchical softmax classifier*: + +- Hierarchical softmax classifier is one of a few solutions to the computational problem. + - Instead of trying to categorize something into all 10,000 categories on one go, imagine if you have one classifier, it tells you is the target word in the first 5,000 words in the vocabulary, or is in the second 5,000 words in the vocabulary, until eventually you get down to classify exactly what word it is, so that the leaf of this tree. + - The main advantage is that instead of evaluating `W` output nodes in the neural network to obtain the probability distribution, it is needed to evaluate only about `log2(W)` nodes. + - In practice, the hierarchical softmax classifier doesn't use a perfectly balanced tree or perfectly symmetric tree. The hierarchical softmax classifier can be developed so that the common words tend to be on top, whereas the less common words like durian can be buried much deeper in the tree. + +*How to sample context `c`*: + +- One thing you could do is just sample uniformly, at random, from your training corpus. + - When we do that, you find that there are some words like `the, of, a, and, to` and so on that appear extremely frequently. + - In your context to target mapping pairs just get these these types of words extremely frequently, whereas there are other words like `orange`, `apple`, and also `durian` that don't appear that often. +- In practice the distribution of words `p(c)` isn't taken just entirely uniformly at random for the training set purpose, but instead there are different heuristics that you could use in order to balance out something from the common words together with the less common words. + +*CBOW*: + +The other version of the Word2Vec model is CBOW, the continuous bag of words model, which takes the surrounding contexts from middle word, and uses the surrounding words to try to predict the middle word. And the algorithm also works, which also has some advantages and disadvantages. + +#### Negative Sampling + +Paper: [Distributed Representations of Words and Phrases and their Compositionality](https://arxiv.org/abs/1310.4546) by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeff Dean. + +Negative sampling is a modified learning problem to do something similar to the Skip-Gram model with a much more efficient learning algorithm. + +- `I want a glass of orange juice to go along with my cereal.` + +- To create a new supervised learning problem: given a pair of words like `orange, juice`, we're going to predict it is a context-target pair or not? + +- First, generate a positive example. Sample a context word, like `orange` and a target word, `juice`, associate them with a label of `1`. + +- Then generate negative examples. Take `orange` and pick another random word from the dictionary for `k` times. + + - Choose large values of k for smaller data sets, like 5 to 20, and smaller k for large data sets, like 2 to 5. + + - In this example, k=4. x=`(context, word)`, y=`target`. + + | context | word | target? | + | --- | --- | --- | + | orange | juice | 1 | + | orange | king | 0 | + | orange | book | 0 | + | orange | the | 0 | + | orange | of | 0 | + +- Compared to the original Skip-Gram model: instead of training all 10,000 of them on every iteration which is very expensive, we're only going to train five, or k+1 of them. k+1 binary classification problems is relative cheap to do rather than updating a 10,000 weights of softmax classifier. + +- *How to choose the negative examples*? + + - One thing you could do is sample it according to the empirical frequency of words in your corpus. The problem is you end up with a very high representation of words like 'the', 'of', 'and', and so on. + - Other extreme method would use `p(w)=1/|V|` to sample the negative examples uniformly at random. This is also very non-representative of the distribution of English words. + - The paper choose a method somewhere in-between: ![negative-sampling-p](../_resources/negative-sampling-p.svg). f(wi) is the observed frequency of word wi. + +#### GloVe word vectors + +Paper: [GloVe: Global Vectors for Word Representation](http://nlp.stanford.edu/pubs/glove.pdf) + +- Xij: #times j appear in context of i. (Think Xij as Xct). + - Xij = Xji. + - If the context is always the word immediately before the target word, then Xij is not symmetric. +- For the GloVe algorithm, define context and target as whether or not the two words appear in close proximity, say within ±10 words of each other. So, Xij is a count that captures how often do words i and j appear with each other or close to each other. +- Model: ![glove-model](../_resources/glove-model.svg). + - 𝜃iTej plays the role of 𝜃tTec in the previous sections. + - We just want to learn vectors, so that their end product is a good predictor for how often the two words occur together. + - There are various heuristics for choosing this weighting function `f` that neither gives these words too much weight nor gives the infrequent words too little weight. + - f(Xij) = 0 if Xij = 0 to make sure 0log0=0 + - One way to train the algorithm is to initialize `theta` and `e` both uniformly random, run gradient descent to minimize its objective, and then when you're done for every word, to then take the average. + - For a given words w, you can have efinal to be equal to the embedding that was trained through this gradient descent procedure, plus `theta` trained through this gradient descent procedure divided by two, because `theta` and `e` in this particular formulation play **symmetric** roles unlike the earlier models we saw in the previous videos, where theta and e actually play different roles and couldn't just be averaged like that. + +Conclusion: + +- The way that the inventors end up with this algorithm was, they were building on the history of much more complicated algorithms like the newer language model, and then later, there came the Word2Vec skip-gram model, and then this came later. +- But when you learn a word embedding using one of the algorithms that we've seen, such as the GloVe algorithm that we just saw on the previous slide, what happens is, you cannot guarantee that the individual components of the embeddings are interpretable. +- But despite this type of linear transformation, the parallelogram map that we worked out when we were describing analogies, that still works. + +### Applications using Word Embeddings + +#### Sentiment Classification + +| comments | stars | +| --- | --- | +| The dessert is excellent. | 4 | +| Service was quite slow. | 2 | +| Good for a quick meal, but nothing special. | 3 | +| Completely lacking in good taste, good service, and good ambience. | 1 | + +*A simple sentiment classification model*: + +![sentiment-model](../_resources/sentiment-model-simple.png) + +- So one of the challenges of sentiment classification is you might not have a huge label data set. +- If this was trained on a very large data set, like a hundred billion words, then this allows you to take a lot of knowledge even from infrequent words and apply them to your problem, even words that weren't in your labeled training set. +- Notice that by using the average operation here, this particular algorithm works for reviews that are short or long because even if a review that is 100 words long, you can just sum or average all the feature vectors for all hundred words and so that gives you a representation, a 300-dimensional feature representation, that you can then pass into your sentiment classifier. +- One of the problems with this algorithm is it **ignores word order**. + - "Completely *lacking* in *good* taste, *good* service, and *good* ambiance". + - This is a very negative review. But the word good appears a lot. + +*A more sophisticated model*: + +![sentiment-model-rnn](../_resources/sentiment-model-rnn.png) + +- Instead of just summing all of your word embeddings, you can instead use a RNN for sentiment classification. + - In the graph, the one-hot vector representation is skipped. + - This is an example of a many-to-one RNN architecture. + +#### Debiasing word embeddings + +Paper: [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://arxiv.org/abs/1607.06520) + +Word embeddings maybe have the bias problem such as gender bias, ethnicity bias and so on. As word embeddings can learn analogies like man is to woman like king to queen. The paper shows that a learned word embedding might output: + +``` +Man: Computer_Programmer as Woman: Homemaker +``` + +Learning algorithms are making very important decisions and so I think it's important that we try to change learning algorithms to diminish as much as is possible, or, ideally, eliminate these types of undesirable biases. + +- *Identify bias direction* + + - The first thing we're going to do is to identify the direction corresponding to a particular bias we want to reduce or eliminate. + - And take a few of these differences and basically average them. And this will allow you to figure out in this case that what looks like this direction is the gender direction, or the bias direction. Suppose we have a 50-dimensional word embedding. + - g1 = eshe \- ehe + - g2 = egirl \- eboy + - g3 = emother \- efather + - g4 = ewoman \- eman + - g = g1 \+ g2 \+ g3 \+ g4 \+ ... for gender vector. + - Then we have + - `cosine_similarity(sophie, g)) = 0.318687898594` + - `cosine_similarity(john, g)) = -0.23163356146` + - to see male names tend to have positive similarity with gender vector whereas female names tend to have a negative similarity. This is acceptable. + - But we also have + - `cosine_similarity(computer, g)) = -0.103303588739` + - `cosine_similarity(singer, g)) = 0.185005181365` + - It is astonishing how these results reflect certain unhealthy gender stereotypes. + - The bias direction can be higher than 1-dimensional. Rather than taking an average, SVD (singular value decomposition) and PCA might help. +- *Neutralize* + + - For every word that is not definitional, project to get rid of bias. + + ![embedding-debiased](../_resources/embedding_debiased.svg) + +- *Equalize pairs* + + - In the final equalization step, what we'd like to do is to make sure that words like grandmother and grandfather are both exactly the same similarity, or exactly the same distance, from words that should be gender neutral, such as babysitter or such as doctor. + - The key idea behind equalization is to make sure that a particular pair of words are equi-distant from the 49-dimensional g⊥. + + ![equalize](../_resources/equalize.png) + + +## Week 3: Sequence models & Attention mechanism + +> Sequence models can be augmented using an attention mechanism. This algorithm will help your model understand where it should focus its attention given a sequence of inputs. This week, you will also learn about speech recognition and how to deal with audio data. + +### Various sequence to sequence architectures + +#### Basic Models + +In this week, you hear about sequence-to-sequence models, which are useful for everything from machine translation to speech recognition. + +- Machine translation + + - Papers: + - [Sequence to Sequence Learning with Neural Networks](https://arxiv.org/abs/1409.3215) by Ilya Sutskever, Oriol Vinyals, Quoc V. Le. + - [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://arxiv.org/abs/1406.1078) by Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio. + - Input a French sentence: `Jane visite l’Afrique en septembre`, we want to translate it to the English sentence: `Jane is visiting Africa in September`. + - First, let's have a network, which we're going to call the encoder network be built as a RNN, and this could be a GRU and LSTM, feed in the input French words one word at a time. And after ingesting the input sequence, the RNN then offers a vector that represents the input sentence. + - After that, you can build a decoder network which takes as input the encoding and then can be trained to output the translation one word at a time until eventually it outputs the end of sequence. + - The model simply uses an encoder network to find an encoding of the input French sentence and then use a decoder network to then generate the corresponding English translation. + + ![translation-seq-seq](../_resources/seq-seq.png) + +- Image Captioning + + - This architecture is very similar to the one of machine translation. + - Paper: [Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)](https://arxiv.org/abs/1412.6632) by Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, Alan Yuille. + - In Course 4, you've seen how you can input an image into a convolutional network, maybe a pre-trained AlexNet, and have that learn an encoding or learn a set of features of the input image. + - In the AlexNet architecture, if we get rid of this final Softmax unit, the pre-trained AlexNet can give you a 4096-dimensional feature vector of which to represent this picture of a cat. And so this pre-trained network can be the **encoder** network for the image and you now have a 4096-dimensional vector that represents the image. You can then take this and feed it to an RNN, whose job it is to generate the caption one word at a time. + + ![image-captioning](../_resources/image-caption.png) + + +#### Picking the most likely sentence + +There are some similarities between the sequence to sequence machine translation model and the language models that you have worked within the first week of this course, but there are some significant differences as well. + +- The machine translation is very similar to a **conditional** language model. + + - You can use a language model to estimate the probability of a sentence. + - The decoder network of the machine translation model looks pretty much identical to the language model, except that instead of always starting along with the vector of all zeros, it has an encoder network that figures out some representation for the input sentence. + - Instead of modeling the probability of any sentence, it is now modeling the probability of the output English translation conditioned on some input French sentence. In other words, you're trying to estimate the probability of an English translation. + + ![mt-as-conditional-lm](../_resources/mt-conditional-lm.png) + +- The difference between machine translation and the earlier language model problem is: rather than wanting to generate a sentence at random, you may want to try to find the most likely English translation. + +- In developing a machine translation system, one of the things you need to do is come up with an algorithm that can actually find the value of y that maximizes p(y<1>,...,y<T_y>|x<1>,...,x<T_x>). The most common algorithm for doing this is called **beam search**. + + - The set of all English sentences of a certain length is too large to exhaustively enumerate. The total number of combinations of words in the English sentence is exponentially larger. So it turns out that the greedy approach, where you just pick the best first word, and then, after having picked the best first word, try to pick the best second word, and then, after that, try to pick the best third word, that approach doesn't really work. + - The most common thing to do is use an approximate search out of them. And, what an approximate search algorithm does, is it will try, it won't always succeed, but it will to pick the sentence, y, that maximizes that conditional probability. + +#### Beam Search + +In the example of the French sentence, `"Jane, visite l'Afrique en Septembre"`. + +- Step 1: pick the first word of the English translation. + + - Set `beam width B = 3`. + - Choose the most likely **three** possibilities for the first words in the English outputs. Then Beam search will store away in computer memory that it wants to try all of three of these words. + - Run the input French sentence through the encoder network and then this first step will then decode the network, this is a softmax output overall 10,000 possibilities (if we have a vocabulary of 10,000 words). Then you would take those 10,000 possible outputs p(y<1>|x) and keep in memory which were the *top three*. + - For example, after this step, we have the three words as `in, Jane, September`. + + ![beam-search-step1](../_resources/beam-search-1.png) + +- Step 2: consider the next word. + + - Find the pair of the first and second words that is most likely it's not just a second where is most likely. By the rules of conditional probability, it's p(y<1>,y<2>|x) = p(y<1>|x) * p(y<2>|x,y<1>). + - After this step, `in september, jane is, jane visit` is left. And notice that `September` has been rejected as a candidate for the first word. + - Because `beam width` is equal to 3, every step you instantiate three copies of the network to evaluate these partial sentence fragments and the output. + - Repeat this step until terminated by the end of sentence symbol. + + ![beam-search-step2](../_resources/beam-search-2.png) + +- If beam width is 1, this essentially becomes the greedy search algorithm. + + +#### Refinements to Beam Search + +- *Length normalization:* + + - Beam search is to maximize the probability: + + ![beam-search-p](../_resources/beam-search-p.svg) + + - But multiplying a lot of numbers less than 1 will result in a very tiny number, which can result in numerical underflow. + + - So instead, we maximizing a log version: + + ![beam-search-logp](../_resources/beam-search-logp.svg) + + - If you have a very long sentence, the probability of that sentence is going to be low, because you're multiplying many terms less than 1. And so the objective function (the original version as well as the log version) has an undesirable effect, that maybe it unnaturally tends to prefer very short translations. It tends to prefer very short outputs. + + - A normalized log-likelihood objective: + + ![beam-search-normalize](../_resources/beam-search-norm.svg) + + - 𝛼 is another hyperparameter + - 𝛼=0 no normalizing + - 𝛼=1 full normalization +- *How to choose beam width B?* + + - If beam width is large: + - consider a lot of possibilities, so better result + - consuming a lot of different options, so slower and memory requirements higher + - If beam width is small: + - worse result + - faster, memory requirements lower + - choice of beam width is application dependent and domain dependent + - In practice, B=10 is common in a production system, whereas B=100 is uncommon. + - B=1000 or B=3000 is not uncommon for research systems. + - But when B gets very large, there is often diminishing returns. +- Unlike exact search algorithms like BFS (Breadth First Search) or DFS (Depth First Search), Beam Search runs faster but is not guaranteed to find exact maximum for argmaxy𝑃(𝑦|𝑥). + + +#### Error analysis in beam search + +- Beam search is an approximate search algorithm, also called a heuristic search algorithm. And so it doesn't always output the most likely sentence. +- In order to know whether it is the beam search algorithm that's causing problems and worth spending time on, or whether it might be the RNN model that's causing problems and worth spending time on, we need to do error analysis with beam search. +- Getting more training data or increasing the beam width might not get you to the level of performance you want. +- You should break the problem down and figure out what's actually a good use of your time. +- *The error analysis process:* + - Problem: + - To translate: `Jane visite l’Afrique en septembre.` (x) + - Human: `Jane visits Africa in September.` (y*) + - Algorithm: `Jane visited Africa last September.` (ŷ) which has some error. + - Analysis: + - Case 1: + + | Human | Algorithm | p(y*\|x) vs p(ŷ\|x) | At fault? | + | --- | --- | --- | --- | + | Jane visits Africa in September. | Jane visited Africa last September. | p(y*\|x) > p(ŷ\|x) | Beam search | + | ... | ... | ... | ... | + + - Case 2: + + | Human | Algorithm | p(y*\|x) vs p(ŷ\|x) | At fault? | + | --- | --- | --- | --- | + | Jane visits Africa in September. | Jane visited Africa last September. | p(y*\|x) ≤ p(ŷ\|x) | RNN | + | ... | ... | ... | ... | + + +#### Bleu Score (optional) + +Paper: [BLEU: a Method for Automatic Evaluation of Machine Translation](https://www.aclweb.org/anthology/P02-1040.pdf) by Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. + +BLEU stands for **bilingual evaluation understudy**. + +- The reason the BLEU score was revolutionary for machine translation was because this gave a pretty good, by no means perfect, but pretty good single real number evaluation metric and so that accelerated the progress of the entire field of machine translation. + +- The intuition behind the BLEU score is we're going to look at the machine generated output and see if the types of words it generates appear in at least one of the human generated references. And so these human generated references would be provided as part of the dev set or as part of the test set. + + - One way to measure how good the machine translation output is to look at each of the words in the output and see if it appears in the references. + - An extreme example: + - French: `Le chat est sur le tapis.` + - Reference 1: `The cat is on the mat.` + - Reference 2: `There is a cat on the mat.` + - MT output: `the the the the the the the.` + - Precision: 7/7. This is not a particularly useful measure because it seems to imply that this MT output has very high precision. + - Instead, what we're going to use is a modified precision measure in which we will give each word credit only up to the maximum number of times it appears in the reference sentences. + - Modified precision: 2/7. The numerator is the count of the number of times the word, `the`, appears. We take a max, we clip this count, at 2. +- In the BLEU score, you don't want to just look at isolated words. You maybe want to look at pairs of words as well. Let's define a portion of the BLEU score on bigrams. + + - MT output: `The cat the cat on the mat.` + + | Bigram | Count | Countclip | + | --- | --- | --- | + | the cat | 2 | 1 | + | cat the | 1 | 0 | + | cat on | 1 | 1 | + | on the | 1 | 1 | + | the mat | 1 | 1 | + | *sum* | 6 | 4 | + + - Modified bigram precision: 4/6 + +- Generally, Bleu score on n-grams is defined as: + + ![ngram-precision](../_resources/bleu-ngram.svg) + +- **Combined Bleu score** = ![bleu](../_resources/bleu-combined.svg) + + - BP is for **brevity penalty**. Preventing short sentences from scoring too high. + - `BP = 1`, if `MT_output_length > reference_output_length`, or + - `BP = exp⁡(1 − reference_output_length / MT_output_length)`, otherwise. + +#### Attention Model Intuition + +Paper: [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/abs/1409.0473) by Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio. + +You've been using an Encoder-Decoder architecture for machine translation. Where one RNN reads in a sentence and then different one outputs a sentence. There's a modification to this called the Attention Model that makes all this work much better. + +The French sentence: + +> Jane s'est rendue en Afrique en septembre dernier, a apprécié la culture et a rencontré beaucoup de gens merveilleux; elle est revenue en parlant comment son voyage était merveilleux, et elle me tente d'y aller aussi. + +The English translation: + +> Jane went to Africa last September, and enjoyed the culture and met many wonderful people; she came back raving about how wonderful her trip was, and is tempting me to go too. + +- The way a human translator would translate this sentence is not to first read the whole French sentence and then memorize the whole thing and then regurgitate an English sentence from scratch. Instead, what the human translator would do is read the first part of it, maybe generate part of the translation, look at the second part, generate a few more words, look at a few more words, generate a few more words and so on. + + ![encoder-decoder](../_resources/encoder-decoder.png) + +- The Encoder-Decoder architecture above is that it works quite well for short sentences, so we might achieve a relatively high Bleu score, but for very long sentences, maybe longer than 30 or 40 words, the performance comes down. (The blue line) + + ![bleu-score-drop](../_resources/bleu-score-drop.png) + +- The Attention model which translates maybe a bit more like humans looking at part of the sentence at a time. With an Attention model, machine translation systems performance can look like the green line above. + + ![attention-intuition](../_resources/attention-intuition.png) + +- What the Attention Model would be computing is a set of attention weights and we're going to use 𝛼<1,1> to denote when you're generating the first words, how much should you be paying attention to this first piece of information here and 𝛼<1,2> which tells us what we're trying to compute the first word of *Jane*, how much attention we're paying to the second word from the inputs, and 𝛼<1,3> and so on. + +- Together this will be exactly the context from, denoted as `C`, that we should be paying attention to, and that is input to the RNN unit to try to generate the first word. + +- In this way the RNN marches forward generating one word at a time, until eventually it generates maybe the `` and at every step, there are **attention weighs** 𝛼<t,t'> that tells it, when you're trying to generate the *t*-th English word, how much should you be paying attention to the *t'*-th French word. + + +#### Attention Model + +- Assume you have an input sentence and you use a bidirectional RNN, or bidirectional GRU, or bidirectional LSTM to compute features on every word. In practice, GRUs and LSTMs are often used for this, maybe LSTMs be more common. The notation for the Attention model is shown below. + + ![attention-model](../_resources/attention-model.png) + +- Compute attention weights: + + ![attention-weights-alpha](../_resources/attention-alpha.svg) + +- Compute e<t,t'> using a small neural network: + + - And the intuition is, if you want to decide how much attention to pay to the activation of t', it seems like it should depend the most on is what is your own hidden state activation from the previous time step. And then a<t'>, the features from time step t', is the other input. + + - So it seems pretty natural that 𝛼<t,t'> and e<t,t'> should depend on s<t-1> and a<t'> . But we don't know what the function is. So one thing you could do is just train a very small neural network to learn whatever this function should be. And trust the backpropagation and trust gradient descent to learn the right function. + + ![attention-weights-alpha-e-nn](../_resources/attention-alpha-e-nn.png) + +- One downside to this algorithm is that it does take quadratic time or quadratic cost to run this algorithm. If you have Tx words in the input and Ty words in the output then the total number of these attention parameters are going to be Tx \* Ty. + +- Visualize the attention weights 𝛼<t,t'>: + + ![visualize-alpha](../_resources/attention-alpha-vis.png) + + +*Implementation tips*: + +- The diagram on the left shows the attention model. +- The diagram on the right shows what one "attention" step does to calculate the attention variables 𝛼<t,t'>. +- The attention variables 𝛼<t,t'> are used to compute the context variable context<t> for each timestep in the output (t=1, ..., Ty). + +
Neural machine translation with attention
+ +### Speech recognition - Audio data + +#### Speech recognition + +- What is the speech recognition problem? You're given an audio clip, x, and your job is to automatically find a text transcript, y. +- So, one of the most exciting trends in speech recognition is that, once upon a time, speech recognition systems used to be built using *phonemes* and this were, I want to say, hand-engineered basic units of cells. + - Linguists use to hypothesize that writing down audio in terms of these basic units of sound called phonemes would be the best way to do speech recognition. +- But with end-to-end deep learning, we're finding that phonemes representations are no longer necessary. But instead, you can built systems that input an audio clip and directly output a transcript without needing to use hand-engineered representations like these. + - One of the things that made this possible was going to much larger data sets. + - Academic data sets on speech recognition might be as a 300 hours, and in academia, 3000 hour data sets of transcribed audio would be considered reasonable size. + - But, the best commercial systems are now trains on over 10,000 hours and sometimes over a 100,000 hours of audio. + +*How to build a speech recognition?* + +- **Attention model for speech recognition**: one thing you could do is actually do that, where on the horizontal axis, you take in different time frames of the audio input, and then you have an attention model try to output the transcript like, "the quick brown fox". + + ![speech-recognition-attention](../_resources/speech-recognition-attention.png) + +- **CTC cost for speech recognition**: Connectionist Temporal Classification + + - Paper: [Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks](https://www.cs.toronto.edu/~graves/icml_2006.pdf) by Alex Graves, Santiago Fernandes, Faustino Gomez, and Jürgen Schmidhuber. + + ![CTC-rnn](../_resources/CTC-rnn.png) + + - For simplicity, this is a simple of what uni-directional for the RNN, but in practice, this will usually be a bidirectional LSTM and bidirectional GRU and usually, a deeper model. But notice that the number of time steps here is very large and in speech recognition, usually the number of input time steps is much bigger than the number of output time steps. + + - For example, if you have 10 seconds of audio and your features come at a 100 hertz so 100 samples per second, then a 10 second audio clip would end up with a thousand inputs. But your output might not have a thousand alphabets, might not have a thousand characters. + - The CTC cost function allows the RNN to generate an output like `ttt_h_eee___[]___qqq__`, here `_` is for "blank", `[]` for "space". + + - The basic rule for the CTC cost function is to collapse repeated characters not separated by "blank". + + +#### Trigger Word Detection + +- With the rise of speech recognition, there have been more and more devices. You can wake up with your voice, and those are sometimes called *trigger word detection systems*. + + ![trigger-word-detection](../_resources/trigger-word-detection.png) + +- The literature on triggered detection algorithm is still evolving, so there isn't wide consensus yet, on what's the best algorithm for trigger word detection. + +- With a RNN what we really do, is to take an audio clip, maybe compute spectrogram features, and that generates audio features x<1>, x<2>, x<3>, that you pass through an RNN. So, all that remains to be done, is to define the target labels y. + + - In the training set, you can set the target labels to be zero for everything before that point, and right after that, to set the target label of one. Then, if a little bit later on, the trigger word was said again at this point, then you can again set the target label to be one. + - Actually it just won't actually work reasonably well. One slight disadvantage of this is, it creates a very imbalanced training set, so we have a lot more zeros than we want. + - One other thing you could do, that it's little bit of a hack, but could make the model a little bit easier to train, is instead of setting only a single time step to operate one, you could actually make it to operate a few ones for several times. *Guide to label the positive/negative words)*: + - Assume labels y<t> represent whether or not someone has just finished saying "activate." + - y<t> = 1 when that that clip has finished saying "activate". + - Given a background clip, we can initialize y<t> = 0 for all `t`, since the clip doesn't contain any "activates." + - When you insert or overlay an "activate" clip, you will also update labels for y<t>. + - Rather than updating the label of a single time step, we will update 50 steps of the output to have target label 1. + - Recall from the lecture on trigger word detection that updating several consecutive time steps can make the training data more balanced. + +*Implementation tips*: + +- Data synthesis is an effective way to create a large training set for speech problems, specifically trigger word detection. +- Using a spectrogram and optionally a 1D conv layer is a common pre-processing step prior to passing audio data to an RNN, GRU or LSTM. +- An end-to-end deep learning approach can be used to build a very effective trigger word detection system. + +* * * + +Notes by Aaron © 2020 \ No newline at end of file diff --git a/DeepLearning_AI/README.md b/DeepLearning_AI/README.md new file mode 100644 index 0000000..72388a3 --- /dev/null +++ b/DeepLearning_AI/README.md @@ -0,0 +1,42 @@ +--- +title: README +updated: 2022-05-16 18:26:58Z +created: 2022-05-16 17:43:56Z +--- + +# Deep Learning Specialization Course Notes + +This is the notes of the [Deep Learning Specialization](https://www.coursera.org/specializations/deep-learning) courses offered by [deeplearning.ai](https://www.deeplearning.ai/) on Coursera. + +Introduction from the specialization page: + +>In five courses, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more. You will work on case studies from healthcare, autonomous driving, sign language reading, music generation, and natural language processing. You will master not only the theory, but also see how it is applied in industry. You will practice all these ideas in Python and in TensorFlow, which we will teach. + +*The Specialization consists of five courses*: + +- [Course 1: Neural Networks and Deep Learning](:/8b8d24c8270944829c58a2071481e8b7) + - [Week 1: Introduction to Deep Learning](:/8b8d24c8270944829c58a2071481e8b7#week-1-introduction-to-deep-learning) + - [Week 2: Neural Networks Basics](:/8b8d24c8270944829c58a2071481e8b7#week-2-neural-networks-basics) + - [Week 3: Shallow Neural Networks](:/8b8d24c8270944829c58a2071481e8b7#week-3-shallow-neural-networks) + - [Week 4: Deep Neural Networks](:/8b8d24c8270944829c58a2071481e8b7#week-4-deep-neural-networks) +- [Course 2: Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization](:/eae2031754c04c308d51a81b8b0e4b1f) + - [Week 1: Practical aspects of Deep Learning](:/eae2031754c04c308d51a81b8b0e4b1f#week-1-practical-aspects-of-deep-learning) + - [Week 2: Optimization algorithms](:/eae2031754c04c308d51a81b8b0e4b1f#week-2-optimization-algorithms) + - [Week 3: Hyperparameter tuning, Batch Normalization and Programming Frameworks](:/eae2031754c04c308d51a81b8b0e4b1f#week-3-hyperparameter-tuning-batch-normalization-and-programming-frameworks) +- [Course 3: Structuring Machine Learning Projects](:/955c977bae464e6aa23cd6f94a8461ed) + - [Week 1: ML Strategy (1)](:/955c977bae464e6aa23cd6f94a8461ed#week-1-ml-strategy-1) + - [Week 2: ML Strategy (2)](:/955c977bae464e6aa23cd6f94a8461ed#week-2-ml-strategy-2) +- [Course 4: Convolutional Neural Networks](:/1c1155ef678f4b41a1b0aa6fd36eabad) + - [Week 1: Foundations of Convolutional Neural Networks](:/1c1155ef678f4b41a1b0aa6fd36eabad#week-1-foundations-of-convolutional-neural-networks) + - [Week 2: Classic Networks](:/1c1155ef678f4b41a1b0aa6fd36eabad#week-2-classic-networks) + - [Week 3: Object detection](:/1c1155ef678f4b41a1b0aa6fd36eabad#week-3-object-detection) + - [Week 4: Special applications: Face recognition & Neural style transfer](:/1c1155ef678f4b41a1b0aa6fd36eabad#week-4-special-applications-face-recognition--neural-style-transfer) +- [Course 5: Sequence Models](:/8c3f644598994759a0d60b2a12997e60) + - [Week 1: Recurrent Neural Networks](:/8c3f644598994759a0d60b2a12997e60#week-1-recurrent-neural-networks) + - [Week 2: Natural Language Processing & Word Embeddings](:/8c3f644598994759a0d60b2a12997e60#week-2-natural-language-processing--word-embeddings) + - [Week 3: Sequence models & Attention mechanism](:/8c3f644598994759a0d60b2a12997e60#week-3-sequence-models--attention-mechanism) + + + +[fancy-course-summary]: https://www.slideshare.net/TessFerrandez/notes-from-coursera-deep-learning-courses-by-andrew-ng +[math-html]: https://www.toptal.com/designers/htmlarrows/letters/ \ No newline at end of file diff --git a/Markdown/Mermaid.md b/Markdown/Mermaid.md new file mode 100644 index 0000000..288b752 --- /dev/null +++ b/Markdown/Mermaid.md @@ -0,0 +1,47 @@ +--- +title: Mermaid +updated: 2022-04-03 12:20:47Z +created: 2022-04-03 12:17:09Z +--- + +```mermaid +graph TD; +A-->B; +A-->C; +B-->D; +C-->D; +``` + +## sequenceDiagram +```mermaid +sequenceDiagram +participant Alice +participant Bob +Alice->John: Hello John, how are you? +loop Healthcheck + John->John: Fight against hypochondria +end +Note right of John: Rational thoughts
prevail... +John-->Alice: Great! +John->Bob: How about you? +Bob-->John: Jolly good! +``` + +## Gantt Charts +```mermaid +gantt +dateFormat YYYY-MM-DD +title Adding GANTT diagram functionality to mermaid +section A section +Completed task :done, des1, 2014-01-06,2014-01-08 +Active task :active, des2, 2014-01-09, 3d +Future task : des3, after des2, 5d +Future task2 : des4, after des3, 5d +section Critical tasks +Completed task in the critical line :crit, done, 2014-01-06,24h +Implement parser and jison :crit, done, after des1, 2d +Create tests for parser :crit, active, 3d +Future task in critical line :crit, 5d +Create tests for renderer :2d +Add to mermaid :1d +``` \ No newline at end of file diff --git a/Overig/Docker/Docker vs lxc.md b/Overig/Docker/Docker vs lxc.md new file mode 100644 index 0000000..8d8d2c3 --- /dev/null +++ b/Overig/Docker/Docker vs lxc.md @@ -0,0 +1,11 @@ +--- +title: Docker vs lxc +updated: 2021-07-26 13:05:45Z +created: 2021-05-04 14:58:11Z +--- + +# Docker vs lxc + +https://archives.flockport.com/lxc-vs-docker/ + +docker inspect --format '{{ .NetworkSettings.IPAddress }}' container_name_or_id diff --git a/Overig/Docker/commands.md b/Overig/Docker/commands.md new file mode 100644 index 0000000..9707189 --- /dev/null +++ b/Overig/Docker/commands.md @@ -0,0 +1,121 @@ +--- +title: commands +updated: 2021-11-20 17:25:07Z +created: 2021-05-04 14:58:11Z +--- + +# Docker + +## remove all stopped containers + +- docker rm $(docker ps -a -q) + +### list and remove images + +- docker images +- docker rmi $(docker images -a -q) + +### Docker lifecycle + +1. [docker create \[OPTIONS\] IMAGE \[COMMAND\] \[ARG...\]](https://docs.docker.com/engine/reference/commandline/create/) create but doesn't start +2. [docker rename CONTAINER NEW_NAME](https://docs.docker.com/engine/reference/commandline/rename/) +3. [docker run \[OPTIONS\] IMAGE \[COMMAND\] \[ARG...\]](https://docs.docker.com/engine/reference/commandline/run/) run and create in one statement +4. [docker rm](https://docs.docker.com/engine/reference/commandline/rm/) + docker image rm dd9acebe0b4d +5. [docker update](https://docs.docker.com/engine/reference/commandline/update/) update configuration of one or more containers + +- docker logs -f + +### docker ipaddress or running container + +docker inspect | grep '"IPAddress"' | head -n 1 + +### three basic commands + +```bash +docker images +docker container <-a> +docker run +``` + +### run tensorflow and jupyter at port 8888 + +docker run --rm -v $(pwd):/tf/convolutional -it -p 8888:8888 tensorflow/tensorflow:latest-jupyter + +### run iterative python program directly + +docker run --rm -v $(pwd):/src --rm python:latest python /src/hello-world.py + +### run iterative python shell + +docker run --rm -it -v $(pwd):/src --rm python:latest python + +### run bash inside python container + +docker run --rm -it -v $(pwd):/src --rm python:latest /bin/bash + +### run a daemon with option -d + +```bash +docker run --rm --name my-postgres -e POSTGRES_PASSWORD=qw12aap -d postgres:latest +docker exec -it my-postgres psql -h localhost -U postgres -d postgres +``` + +### docker files + +```dockerfile +FROM python:latest +RUN pip3 install numpy +CMD python3 /src/hello-world.py +``` + +### docker networks + +Usage: docker network COMMAND + +Commands: +connect Connect a container to a network +create Create a network +disconnect Disconnect a container from a network +inspect Display detailed information on one or more networks +ls List networks +prune Remove all unused networks +rm Remove one or more networks + +```bash +docker network create net_1 +docker run --rm -d --net net_1 --name my_py -v $(pwd):/src python:latest python3 /src/run.py +docker run --rm -it --net net_1 alpine:latest /bin/bash + +docker network create net_2 +docker run --rm --name my-postgres --network net_2 -e POSTGRES_PASSWORD=qw12aap -d postgres:latest +docker run -it --rm --name my_postgre2 --network net_2 postgres:latest /bin/bash +``` + +inside: psql -U postgres -h my-postgres + +### Docker Compose + +```docker-compose +version: '3' +services: + python: + image: python:latest + container_name: my_py + volumes: + - .:/src + command: python3 /src/run.py + restart: always + postgres: + image: postgres:latest + container_name: my_post + environment: + - e POSTGRES_PASSWORD=qw12aap + restart: always + alpine: + image: alpine:latest + command: echo "hello from alpine" + restart: always +``` + +[How To Remove Docker Images, Containers, and Volumes](https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes) \ No newline at end of file diff --git a/Overig/Linux/Akonadi server.md b/Overig/Linux/Akonadi server.md new file mode 100644 index 0000000..eae79de --- /dev/null +++ b/Overig/Linux/Akonadi server.md @@ -0,0 +1,9 @@ +--- +title: Akonadi server +updated: 2022-06-04 22:22:09Z +created: 2022-06-04 22:21:49Z +--- + +## Akonadi server + +\[https://userbase.kde.org/Akonadi/nl\](https://userbase.kde.org/Akonadi/nl) \ No newline at end of file diff --git a/Overig/Linux/Bash.md b/Overig/Linux/Bash.md new file mode 100644 index 0000000..c4d1499 --- /dev/null +++ b/Overig/Linux/Bash.md @@ -0,0 +1,72 @@ +--- +title: Bash +updated: 2022-04-27 17:38:16Z +created: 2021-05-04 14:58:11Z +--- + +# Bash + +PROMPT_COMMAND='echo -n "writing the prompt at " && date' + +HISTTIMEFORMAT='I ran this at: %d/%m/%y %T ' + +## CDPATH + +As with the PATH variable, the CDPATH variable is a colon-separated list of paths. When you run a cd command with a relative path (ie one without a leading slash), by default the shell looks in your local folder for matching names. CDPATH will look in the paths you give it for the directory you want to change to. + +If you set CDPATH up like this: + +```bash +CDPATH=/:/lib +``` + +Then typing in: + +```bash +cd /home +cd tmp +``` + +will always take you to /tmp no matter where you are. + +Watch out, though, as if you don’t put the local (.) folder in the list, then you won’t be able to create any other tmp folder and move to it as you normally would: + +$ cd /home +$ mkdir tmp +$ cd tmp +$ pwd +/tmp + +Oops! + +This is similar to the confusion I felt when I realised the dot folder was not included in my more familiar PATH variable… but you should do that in the PATH variable because you can get tricked into running a ‘fake’ command from some downloaded code. + +Correct way: + +```bash +CDPATH=.:/space:/etc:/var/lib:/usr/share:/opt +``` + +### Restart terminal + +```bash +exec "$SHELL" +``` + +### SHLVL + +This variable tracks how deeply nested you are in the bash shell. + +```bash +echo $SHLVL +``` + +### LINENO + +Reports the number of commands that have been run in the session so far. + +### TMOUT + +If nothing is typed in for the number of seconds this is set to, then the shell will exit. + +[direnv](https://github.com/direnv/direnv) \ No newline at end of file diff --git a/Overig/Linux/Ethernet.md b/Overig/Linux/Ethernet.md new file mode 100644 index 0000000..01dcc6a --- /dev/null +++ b/Overig/Linux/Ethernet.md @@ -0,0 +1,46 @@ +--- +title: Ethernet +updated: 2022-04-27 17:47:40Z +created: 2021-05-04 14:58:11Z +--- + +# Ethernet + +## Change mac address: + +macchanger -r + +## local ip address: + +hostname -I + +## public ip address: + +curl ipinfo.io/ip + +## scan local network + +sudo arp-scan --interface=enp4s0 --localnet + +```bash +sudo service network-manager restart +sudo vi [/etc/dhcp/dhclient.conf](file:///etc/dhcp/dhclient.conf) +``` + +edit "prepend domain-name-servers" for DNS servers + +## Monitor wifi + +wavemon + +## available wifi networks + +nmcli connection show + +## which DNS server in use: + +( nmcli dev list || nmcli dev show ) 2>/dev/null | grep DNS +nm-tool | grep DNS + + +[Unbound](https://aacable.wordpress.com/2019/12/10/short-notes-for-unbound-caching-dns-server-under-ubuntu-18/) \ No newline at end of file diff --git a/Overig/Linux/Firewall-1.md b/Overig/Linux/Firewall-1.md new file mode 100644 index 0000000..85c7587 --- /dev/null +++ b/Overig/Linux/Firewall-1.md @@ -0,0 +1,9 @@ +--- +title: Firewall +updated: 2022-06-03 21:51:40Z +created: 2022-06-03 21:51:30Z +--- + + + +https://www.cyberciti.biz/faq/how-to-delete-a-ufw-firewall-rule-on-ubuntu-debian-linux/ \ No newline at end of file diff --git a/Overig/Linux/Firewall.md b/Overig/Linux/Firewall.md new file mode 100644 index 0000000..8e4f152 --- /dev/null +++ b/Overig/Linux/Firewall.md @@ -0,0 +1,15 @@ +--- +title: Firewall +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Firewall + +sudo ufw status verbose +sudo ufw enable +sudo ufw disable + +sudo ufw --help + +gufw diff --git a/Overig/Linux/Keychron Keyboard.md b/Overig/Linux/Keychron Keyboard.md new file mode 100644 index 0000000..59bc5af --- /dev/null +++ b/Overig/Linux/Keychron Keyboard.md @@ -0,0 +1,7 @@ +--- +title: Keychron Keyboard +updated: 2022-04-27 17:37:12Z +created: 2022-04-27 17:36:57Z +--- + +https://gist.github.com/andrebrait/961cefe730f4a2c41f57911e6195e444 \ No newline at end of file diff --git a/Overig/Linux/Linux Kernel.md b/Overig/Linux/Linux Kernel.md new file mode 100644 index 0000000..d149686 --- /dev/null +++ b/Overig/Linux/Linux Kernel.md @@ -0,0 +1,8 @@ +--- +title: Linux Kernel +updated: 2022-04-27 17:41:37Z +created: 2022-04-27 17:41:13Z +--- + +[Linux Kernel Boot Parameters](http://redsymbol.net/linux-kernel-boot-parameters//) + diff --git a/Overig/Linux/Locale.md b/Overig/Linux/Locale.md new file mode 100644 index 0000000..11f77f3 --- /dev/null +++ b/Overig/Linux/Locale.md @@ -0,0 +1,13 @@ +--- +title: Locale +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Locale + +sudo dpkg-reconfigure locales + +- select en_US.UTF-8 +- select en_US.UTF-8 +- type "locale" to check again diff --git a/Overig/Linux/Packages.md b/Overig/Linux/Packages.md new file mode 100644 index 0000000..43cdc6f --- /dev/null +++ b/Overig/Linux/Packages.md @@ -0,0 +1,106 @@ +--- +title: Packages +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Remove unused + +## install + +sudo apt install deborphan + +## run terminal version + +deborphan + +## To remove the orphaned packages, run: + +sudo orphaner + +## graphical version + +sudo apt install gtkorphan +sudo gtkorphan + +## Alternative + +sudo apt autoclean && sudo apt autoremove +dpkg --remove + +## check broken dependencies + +sudo apt-get check + +## search + +apt-cache search [string1 stringn] + +## list all available packages and search + +apt-cache pkgnames +apt-cache search + +## get package info + +apt-cache show + +## dependencies + +apt-cache showpkg + +## statistics + +apt-cache stats + +## distro update + +sudo apt-get dist-upgrade +sudo apt-get install --only-upgrade + +## install specific version + +sudo apt-get install vsftpd=2.3.5-3ubuntu1 + +## remove without removing configurations + +sudo apt-get remove + +## remove configurations + +sudo apt-get purge + +## remove package and configuration + +sudo apt-get remove --purge + +## cleanup diskspace + +sudo apt-get clean + +## check log off a package + +sudo apt-get change log + +## Get Debian version + +lsb_release -a + +## Disk info + +sudo hdparm -I /dev/sda1 + +## which kernels are installed? + +dpkg --list | grep linux-image + +sudo dpkg -i + +sudo dpkg --remove + +sudo add-apt-repository 'deb ./' + +## Create and install rpm installation package + +sudo alien packagename.rpm +sudo dpkg -i packagename.deb diff --git a/Overig/Linux/Search find.md b/Overig/Linux/Search find.md new file mode 100644 index 0000000..5aef63a --- /dev/null +++ b/Overig/Linux/Search find.md @@ -0,0 +1,9 @@ +--- +title: Search find +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Search find + +find /. -name 'toBeSearched.file' 2>&1 | grep -v 'Permission denied' diff --git a/Overig/Linux/Services.md b/Overig/Linux/Services.md new file mode 100644 index 0000000..a87bb8e --- /dev/null +++ b/Overig/Linux/Services.md @@ -0,0 +1,26 @@ +--- +title: Services +updated: 2022-04-27 17:35:25Z +created: 2021-05-04 14:58:11Z +--- + +# Services + +## list all services + +service --status-all + +## start - stop - pauze services + +sudo service 'https://......' +``` + +### If problems with audio and video + +```bash +youtube-dl -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/bestvideo+bestaudio' --merge-output-format mp4 'http://....' +``` + +[More info](https://askubuntu.com/questions/486297/how-to-select-video-quality-from-youtube-dl) + + +ffmpeg -v 5 -y -i input.m4a -acodec libmp3lame -ac 2 -ab 192k output.mp3 \ No newline at end of file diff --git a/Overig/Privacy.md b/Overig/Privacy.md new file mode 100644 index 0000000..a07c333 --- /dev/null +++ b/Overig/Privacy.md @@ -0,0 +1,7 @@ +--- +title: Privacy +updated: 2022-04-27 17:42:36Z +created: 2022-04-27 17:42:29Z +--- + +https://oisd.nl/ \ No newline at end of file diff --git a/Overig/Remove string pdf.md b/Overig/Remove string pdf.md new file mode 100644 index 0000000..af17125 --- /dev/null +++ b/Overig/Remove string pdf.md @@ -0,0 +1,24 @@ +--- +title: Remove string pdf +updated: 2021-10-04 13:26:02Z +created: 2021-10-04 13:21:30Z +latitude: 52.09370000 +longitude: 6.72510000 +altitude: 0.0000 +--- + +```bash +sudo apt install pdftk +``` + +```bash +pdftk file.pdf output uncompressed.pdf uncompress +``` + +```bash +sed -i 's/RemoveString//g' uncompressed.pdf +``` + +```bash +pdftk uncompressed.pdf output changed.pdf compress +``` \ No newline at end of file diff --git a/Overig/kubernetes/CheatSheet K8s.md b/Overig/kubernetes/CheatSheet K8s.md new file mode 100644 index 0000000..55f7919 --- /dev/null +++ b/Overig/kubernetes/CheatSheet K8s.md @@ -0,0 +1,49 @@ +--- +title: CheatSheet K8s +updated: 2022-02-05 17:47:17Z +created: 2022-02-05 15:06:57Z +latitude: 51.86290000 +longitude: 4.61910000 +altitude: 0.0000 +--- + +## minikube +`minikube start | status | stop | delete | pause | unpause | dashboard` + +## Crud commands kubectl +`kubectl create deployment [name]` +`kubectl create deployment nginx-depl --image=nginx` +`kubectl edit deployment [name]` +`kubectl edit deployment nginx-depl` +`kubectl delete deployment [name]` + +## Use configuration file for CRUD +`kubectl apply -f [file name]` +`kubectl delete -f [file name]` + +## Status of different K8s components +`kubectl get nodes | pod | service | replicaset | deployment | all` +`kubectl get pod -o wide` More columns with info +`kubectl get deployment [name] -o yaml` current in yaml format (useful for debugging) + +## Debugging pods +`kuberctl logs [pod name]` +`kubectl logs nginx-depl-{pod-name}` + +`kubectl exec -it [pod name] -- bin/bash` +`kuberctl describe pod [pod name]` +`kuberctl describe pod nginx-depl-{pod name}` + +## Configuration file Deployment & Service +Each configuration file has 3 parts +1. metadata +2. specification +3. status (automatic generated) + - desired state + - Actual state + - Both have to match. If not K8s knows something has to be fixed + - status is updated continues + - etcd is providing the info for status +![99bd030fd518baa2ca26a85b313e23f0.png](../../_resources/99bd030fd518baa2ca26a85b313e23f0.png) + +![b138f59fcc154b98b815a2dc79b89003.png](../../_resources/b138f59fcc154b98b815a2dc79b89003.png) \ No newline at end of file diff --git a/Overig/kubernetes/Ingress.md b/Overig/kubernetes/Ingress.md new file mode 100644 index 0000000..0b3b951 --- /dev/null +++ b/Overig/kubernetes/Ingress.md @@ -0,0 +1,26 @@ +--- +title: Ingress +updated: 2022-02-06 17:57:21Z +created: 2022-02-06 17:38:09Z +latitude: 51.86290000 +longitude: 4.61910000 +altitude: 0.0000 +--- + +# install Ingress addon +Installs an additional pod +Starts automatically th K8s Mginx implimentation of Ingress controller +`minikube addons enable ingress' + +Then create Ingress rule +![77e6428cf3d2bff19c7215e2915ca3c4.png](../../_resources/77e6428cf3d2bff19c7215e2915ca3c4.png) + +or + +![a6f83a73449bec9c309c2cab27c24f5e.png](../../_resources/a6f83a73449bec9c309c2cab27c24f5e.png) + +SSL implementation +![47073357d36804a22ad703c90cb541ce.png](../../_resources/47073357d36804a22ad703c90cb541ce.png) + +Take care!!! +![b97bd5343f507eb8529bc2c00d30084d.png](../../_resources/b97bd5343f507eb8529bc2c00d30084d.png) \ No newline at end of file diff --git a/Overig/kubernetes/Intro Basis Components of K8s.md b/Overig/kubernetes/Intro Basis Components of K8s.md new file mode 100644 index 0000000..c13c373 --- /dev/null +++ b/Overig/kubernetes/Intro Basis Components of K8s.md @@ -0,0 +1,68 @@ +--- +title: Intro Basis Components of K8s +updated: 2022-02-05 15:10:02Z +created: 2022-01-29 15:33:56Z +latitude: 52.38660000 +longitude: 5.27820000 +altitude: 0.0000 +--- + +What is Kubernetes (K8s) + +- container orchestration tool +- manages containers (not only Docker) +- different environments + - physical + - virtual + - cloud + +What problems solves Kubernetes: + +- manages many containers with independent services, like micro services + +Kubernetes features + +- High Availability (no downtime) +- Scalability (high performance) +- Disaster recovery (backup and restore) + +Kubernetes components (but many more) + +- Node (physical or virtual) + - node contains pods and per pod usually 1 application + - pod is smallest unit of K8s + - pod is abstraction over container + - each pod gets its own ip-address + - new ip-address on re-creation + - ![aca7155a4c7e4f2b982378cbb2d37400.png](../../_resources/aca7155a4c7e4f2b982378cbb2d37400.png) +- Services + - permanent ip-address + - lifecycle of Pod and Service are not connected + - ![ad9e649d35df7e77ce73056bbaf2cbb9.png](../../_resources/ad9e649d35df7e77ce73056bbaf2cbb9.png) +- External Service + - opens the communication for external sources + - internal service for eq database. Not accessible from outside + - External request go first to Ingress (to route traffic) + - ![25f981495ea54faccfe5ea58cd1f8b75.png](../../_resources/25f981495ea54faccfe5ea58cd1f8b75.png) +- ConfigMap & Secret + - external configuration of application (eq URL of a database DB_URL = mongo-db-service) + - secret credentials in Secret (like configMap but base64 encoded) + - Application can read from ConfigMap and Secret +- Volumes + - for persistent data + - attach physical storage to a pod + - local machine or remote storage (eq cloud and outside k8s cluster) + - k8s does not manage data persistence +- Deployment and Stateful Set + - Every Pod is replicated as specified in BluePrint + - Service has 2 functionalities: + - Permanent IP (so an other pod can connect of the pod dies) + - load balancer + - the pod is not created, but for every pod is specified (Blueprint) how many replicas are required + - Blueprint is abstraction of Pods. Pods are an abstraction of Containers + - In practice just working with Deployments + - Stateful Set + - Databases have state and can not be replicated + - For stateful applications like databases + - no database inconsistency can occur. + - is not easy therefor advice to host databases outside K8s cluster \ No newline at end of file diff --git a/Overig/kubernetes/K8s Architecture.md b/Overig/kubernetes/K8s Architecture.md new file mode 100644 index 0000000..d12b3dd --- /dev/null +++ b/Overig/kubernetes/K8s Architecture.md @@ -0,0 +1,47 @@ +--- +title: K8s Architecture +updated: 2022-01-29 16:47:31Z +created: 2022-01-29 16:19:16Z +latitude: 52.38660000 +longitude: 5.27820000 +altitude: 0.0000 +--- + +Worker Machine in K8s cluster +- Nodes do the actual work +- each node has multiple Pots on it +- 3 process must be installed on every Node + - Container Runtime (eq Docker) + - Kubelet + - interacts with container and node + - starts the Pod and the container inside + - assigns resources from Node to the container + - Communication between Nodes through Services + - Kube Proxy + - forwards the requests + +Master Node (usally multple master nodes) +- 4 processes that must un on every master Node that control the master state and the workers + - API Server(client interacts with this api) + - acts as a load balances for the master nodes + - require much less resources then worker nodes + - more worker nodes, then more master nodes => makes the application more robust + - cluster gateway + - gets initial request like update or query + - acts as a gatekeeper for authentication + - validates a request and when OK it forwards the request to other processes and eventually to the Pod. + - One entry point to the cluster + - Scheduler + - API Server sends request to Scheduler and will start a Pod + - Has intelligence with Pod has to complete the work (how much cpu, ram etc) + - Kubelet gets the request and executes the request + - Controller manager + - detects cluster state changes like crashes + - tries to recover the state by making a request to the Scheduler + - etcd (key value store off the cluster state) (cluster brain) + - other 3 components get the information from etcd + - does not contain any application data! + - distributed across all the master nodes + ![4c5a7cb8f10e6e2cc67ccfb4681679b7.png](../../_resources/4c5a7cb8f10e6e2cc67ccfb4681679b7.png) + + \ No newline at end of file diff --git a/Python/Algemeen.md b/Python/Algemeen.md new file mode 100644 index 0000000..6950d9f --- /dev/null +++ b/Python/Algemeen.md @@ -0,0 +1,11 @@ +--- +title: Algemeen +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# General Python + +[Install Python](https://tecadmin.net/install-python-3-8-ubuntu/) + +[Use Poetry: Python packaging and dependency management](https://python-poetry.org/) \ No newline at end of file diff --git a/Python/Algemeen/Jupyter Built-in magic commands.md b/Python/Algemeen/Jupyter Built-in magic commands.md new file mode 100644 index 0000000..ba69a05 --- /dev/null +++ b/Python/Algemeen/Jupyter Built-in magic commands.md @@ -0,0 +1,93 @@ +--- +title: Jupyter Built-in magic commands +updated: 2022-04-26 11:13:12Z +created: 2022-04-26 10:31:22Z +--- + +## references + +%lsmagic # list of all magic methods +%quickref # cheatsheet +%magic +%lsmagic + +## Timeit + +%%timeit -n 3 +%time + +The %timeit magic runs the given code many times, then returns the speed of the fastest result. + +``` +%timeit sum(range(100000)) +``` + +The %%timeit cell magic can be used to time blocks of code. Start cell + +| Options | Description | +| --- | --- | +| -n<N> | It executes the code statement <N>times in a loop. If the number is not given, it determines the to get good accuracy. | +| -r<R> | Shows the number of repeats. | +| -p<P> | Used to calculate the precision of <P>digits to show the timing result. | +| -c | Use time.clock; default function on Windows to measure the wall time. | +| -t | Use time.time; the default function on Unix measures the wall time. | +| -q | Use for Quiet; do not display any result. | +| -o | Returns the TimeitResult that is further stored in a variable to view more details | + +``` +%%timeit -n 3 +a = 0 +for i in range(100000): + a += i +``` + +The %time magic times a single run of a function + +``` +%time sum(range(100000)) +``` + +## run scripts + +``` +%run somescript.py +%run -d myscript.py # debug +``` + +## reset kernel + +%reset is not a kernel restart + +## Matplotlib + +``` +from matplotlib import pyplot as plt +%matplotlib inline +``` + +%matplotlib # set matplotlib to work interactively; does not import anythig +%matplotlib inline +%matplotlib qt # request a specific GUI backend + +## debugging + +%debug # jump into the Python debugger (pdb) +%pdb # start the debugger on any uncaught exception. + +%cd # change directory +%pwd # print working directory +%env # OS environment variables + +## OS command + +!OScommand +!ping www.bbc.co.uk +%alias # system command alias +!!date # output \['Sat Jan 19 02:53:54 UTC 2019'\] +%system date + +## Auto reload + +%load_ext autoreload +%autoreload +When you are working with external tools or changing the enviornment variables, this will certainly help you a lot. The external commands help you autoreload the tools and libraries at a specified defined interval. So whenever there is even a minor change, we do not have to run the imports to update the local enviornment of the notebook \ No newline at end of file diff --git a/Python/Algemeen/RE.md b/Python/Algemeen/RE.md new file mode 100644 index 0000000..3982099 --- /dev/null +++ b/Python/Algemeen/RE.md @@ -0,0 +1,46 @@ +--- +title: RE +updated: 2022-04-25 11:42:43Z +created: 2022-04-25 11:29:11Z +--- + + +### giving a label and looking at the results as a dictionary is pretty useful. For that we use the syntax (?P), where the parethesis starts the group, the ?P indicates that this is an extension to basic regexes, and is the dictionary key we want to use wrapped in <>. +```python +for item in re.finditer("(?P[\w ]*)(?P<edit_link>\[edit\])",wiki): + # We can get the dictionary returned for the item with .groupdict() + print(item.groupdict()['edit_link']) +``` + +## Look-ahead and Look-behind +### One more concept to be familiar with is called "look ahead" and "look behind" matching. In this case, the pattern being given to the regex engine is for text either before or after the text we are trying to isolate. For example, in our headers we want to isolate text which comes before the [edit] rendering, but we actually don't care about the [edit] text itself. Thus far we have been throwing the [edit] away, but if we want to use them to match but don't want to capture them we could put them in a group and use look ahead instead with ?= syntax +```python +for item in re.finditer("(?P<title>[\w ]+)(?=\[edit\])",wiki): + # What this regex says is match two groups, the first will be named and called title, will have any amount + # of whitespace or regular word characters, the second will be the characters [edit] but we don't actually + # want this edit put in our output match objects + print(item) +``` + +## I'll actually use this example to show you the verbose mode of python regexes. The verbose mode allows you to write multi-line regexes and increases readability. For this mode, we have to explicitly indicate all whitespace characters, either by prepending them with a \ or by using the \s special value. However, this means we can write our regex a bit more like code, and can even include comments with +```python +pattern=""" +(?P<title>.*) #the university title +(–\ located\ in\ ) #an indicator of the location +(?P<city>\w*) #city the university is in +(,\ ) #separator for the state +(?P<state>\w*) #the state the city is located in""" + +# Now when we call finditer() we just pass the re.VERBOSE flag as the last parameter, this makes it much +# easier to understand large regexes! +for item in re.finditer(pattern,wiki,re.VERBOSE): + # We can get the dictionary returned for the item with .groupdict() + print(item.groupdict()) +``` + +### lets create a pattern. We want to include the hash sign first, then any number of alphanumeric characters. And we end when we see some whitespace +```python +pattern = '#[\w\d]*(?=\s)' +# Notice that the ending is a look ahead. +re.findall(pattern, health) +``` \ No newline at end of file diff --git a/Python/Algemeen/Time Date.md b/Python/Algemeen/Time Date.md new file mode 100644 index 0000000..84eeb08 --- /dev/null +++ b/Python/Algemeen/Time Date.md @@ -0,0 +1,27 @@ +--- +title: Time Date +updated: 2022-04-25 10:57:17Z +created: 2022-04-25 10:53:38Z +--- + +```python +import datetime as dt +import time as tm +``` + +### seconds sinds 1 jan 1970 +```python +tm.time() +``` +Output +1650884042.68 + +```python +dt.datetime.fromtimestamp(tm.time()) +``` +Output: +datetime.datetime(2022, 4, 25, 12, 54, 44, 69803) + +```python +dt.timedelta(days = 100) # create a timedelta of 100 days +``` diff --git a/Python/Algemeen/csv.md b/Python/Algemeen/csv.md new file mode 100644 index 0000000..3cfd38b --- /dev/null +++ b/Python/Algemeen/csv.md @@ -0,0 +1,28 @@ +--- +title: csv +updated: 2022-04-25 10:47:01Z +created: 2022-04-25 10:45:25Z +--- + +```python +import csv + +%precision 2 + +with open('../datasets/mpg.csv') as csvfile: + mpg = list(csv.DictReader(csvfile)) +``` +output gives dict with key the row number: +[{'': '1', + 'manufacturer': 'audi', + 'model': 'a4', + 'displ': '1.8', + 'year': '1999', + 'cyl': '4', + 'trans': 'auto(l5)', + 'drv': 'f', + 'cty': '18', + 'hwy': '29', + 'fl': 'p', + 'class': 'compact'}, + {'': '2', diff --git a/Python/Conda/Installation.md b/Python/Conda/Installation.md new file mode 100644 index 0000000..485a00a --- /dev/null +++ b/Python/Conda/Installation.md @@ -0,0 +1,51 @@ +--- +title: Installation +updated: 2022-07-11 20:02:16Z +created: 2021-05-04 14:58:11Z +--- + +# Conda + +## Activate / Deactivate Conda +```bash +conda config --set auto_activate_base false +conda config --set auto_activate_base true +``` + +conda search [packagename] + +conda install [packagename] + +conda update conda + +### Environments + +conda info --envs + +conda remove --name ENVNAME --all + +conda create --name ENVNAME [optional packagename] optional [python=3.9] + +conda env remove -n ENVNAME + +conda activate ENVNAME + + +[conda-forge](https://conda-forge.org/) +conda install -c conda-forge jupyterlab + + + +## Tensorflow +Just install tensorflow with GPU and all set. +No need anymore to install cuda, because this is installed together with TF python library + + +## Pip +### remove all pip lib +pip freeze --local > reqs.txt +pip uninstall -y -r reqs.txt + + +conda install -c conda-forge matplotlib ipykernel ipython pandas scikit-learn + diff --git a/Python/Conda/Syntax.md b/Python/Conda/Syntax.md new file mode 100644 index 0000000..f76c24c --- /dev/null +++ b/Python/Conda/Syntax.md @@ -0,0 +1,34 @@ +--- +title: Syntax +updated: 2022-06-19 10:43:57Z +created: 2021-11-01 17:08:37Z +latitude: 52.38660000 +longitude: 5.27820000 +altitude: 0.0000 +--- + +# Syntax + +conda info + +conda list \[-n ENV_NAME\] +conda list --revisions + +conda search "^openCV$" +conda search "openCV" + +conda install \[-c conda-forge\] <package-name>=x.y +conda install --revision 2</package-name> + +conda update \[-n ENV_NAME\] --all +conda remove package-name + +conda create -n ENV_NAME \[python=x.x\] ipython +conda create --name ENV\_NAME\_1 --clone ENV\_NAME\_2 + +conda env list +conda activate ENV_NAME +conda env remove -n ENV_NAME +conda remove -n ENV_NAME --all + +`conda install -c conda-forge ipykernel pandas` \ No newline at end of file diff --git a/Python/Conda/Tensorflow.md b/Python/Conda/Tensorflow.md new file mode 100644 index 0000000..ba266e1 --- /dev/null +++ b/Python/Conda/Tensorflow.md @@ -0,0 +1,34 @@ +--- +title: Tensorflow +updated: 2022-06-29 11:34:19Z +created: 2022-06-03 16:38:35Z +--- + + + + + +conda create -n tf + + +conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0 + +pip install tensorflow + + + +conda install mamba -n base -c conda-forge +mamba repoquery whoneeds ipython [-t] +[source Mamba](https://github.com/mamba-org/mamba) + + +conda install -c anaconda tensorflow-gpu +conda install tensorflow-gpu -c conda-forge + + + + + + + +[source Tensorflow](https://www.tensorflow.org/install/pip) diff --git a/Python/Decorators.md b/Python/Decorators.md new file mode 100644 index 0000000..ccf7a96 --- /dev/null +++ b/Python/Decorators.md @@ -0,0 +1,21 @@ +--- +title: Decorators +updated: 2022-04-03 12:04:11Z +created: 2021-05-04 14:58:11Z +--- + +# Decorators + +[Decorators explained](https://lerner.co.il/2019/05/05/making-your-python-decorators-even-better-with-functool-wraps/) + +```python +import attr + +@attr.s(auto_attribs=True) +class Book(object): + isbn: str + name: str + author: str + published_year: int + edition: int +``` \ No newline at end of file diff --git a/Python/Docker.md b/Python/Docker.md new file mode 100644 index 0000000..9a6b50b --- /dev/null +++ b/Python/Docker.md @@ -0,0 +1,9 @@ +--- +title: Docker +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Deply to docker + +[Faster Docker builds with pipenv, poetry, or pip-tools](https://pythonspeed.com/articles/pipenv-docker/) \ No newline at end of file diff --git a/Python/ES.md b/Python/ES.md new file mode 100644 index 0000000..e6697d8 --- /dev/null +++ b/Python/ES.md @@ -0,0 +1,30 @@ +--- +title: ES +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# ElasticSearch + +## Ping a ES server + +``` python +from elasticsearch import Elasticsearch +from elasticsearch.exceptions import ConnectionError + +try: + es:bool = Elasticsearch([{'host': 'localhost', 'port': 9200}]).ping() + print(es) +except ConnectionRefusedError: + print ('Connection Error!') +``` + +## get default response + +``` python +import requests +r:requests.models.Response = requests.get('http://localhost:9200') +print(r) +r.headers +r.content +``` diff --git a/Python/JSON.md b/Python/JSON.md new file mode 100644 index 0000000..9ad81f7 --- /dev/null +++ b/Python/JSON.md @@ -0,0 +1,48 @@ +--- +title: JSON +updated: 2022-04-03 12:15:44Z +created: 2022-01-17 17:27:47Z +latitude: 52.38660000 +longitude: 5.27820000 +altitude: 0.0000 +--- + +# JSON + +```python +import json +from types import SimpleNamespace + +# Initialize JSON data +json_data = '[{"studentid": 1, "name": "Nikhil", "subjects":\ +["Python", "Data Structures"], "company":"GFG"}, \ +{"studentid": 2, "name": "Nisha", "subjects":\ +["Java", "C++", "R Lang"], "company":"GFG"}]' + +x = json.loads(json_data, object_hook=lambda d: SimpleNamespace(**d)) +print(x[0].subjects) +``` + + +```python +class Employee(object): +def __init__(self,name1, name2): +self.name1 = name1 +self.name2 = name2 +def toJSON(self): +return json.dumps(self, default=lambda o: o.__dict__, +sort_keys=True, indent=4) + + +ader = Employee('Olifant', 'aap') +json.loads(ader.toJSON())['name2'] +``` + + +```mermaid +graph TD; +A-->B; +A-->C; +B-->D; +C-->D; +``` \ No newline at end of file diff --git a/Python/Jupyter.md b/Python/Jupyter.md new file mode 100644 index 0000000..9f2e206 --- /dev/null +++ b/Python/Jupyter.md @@ -0,0 +1,63 @@ +--- +title: Jupyter +updated: 2021-12-20 09:04:10Z +created: 2021-05-04 14:58:11Z +--- + +# Jupyter + +```bash +pip install jupyterthemes +``` + +```bash +jt -l # list available themes +jupyter-kernelspec list +jupyter --paths # will list all of the possible locations for everything it uses to run: kernels, extensions, pidfiles, etc. +``` + +```bash +jupyter notebook --ip=0.0.0.0 --port=8080 --no-browser +jupyter lab --ip=0.0.0.0 --port=8080 --no-browser +``` + +```bash +export PYSPARK_DRIVER_PYTHON="/home/john/environments/notebook/bin/ipython" +``` + +```python +# Import findspark when not pip install pyspark +import findspark + +# Initialize and provide path +findspark.init("/home/john/opt/spark") +# or let spark find it +findspark.init() +``` + +https://towardsdatascience.com/introducing-jupytext-9234fdff6c57 +https://github.com/mwouts/jupytext + +|Command|Description| +---|---| +|%quickref| Display the IPython Quick Reference Card| +|%magic| Display detailed documentation for all of the available magic commands| +|%debug| Enter the interactive debugger at the bottom of the last exception traceback| +|%hist| Print command input (and optionally output) history| +|%pdb| Automatically enter debugger after any exception| +|%paste| Execute preformatted Python code from clipboard| +|%cpaste| Open a special prompt for manually pasting Python code to be executed| +|%reset| Delete all variables/names defined in interactive namespace| +|%page| OBJECT Pretty-print the object and display it through a pager| +|%run script.py| Run a Python script inside IPython| +|%prun| statement Execute statement with cProfile and report the profiler output| +|%time| statement Report the execution time of a single statement| +|%timeit| statement Run a statement multiple times to compute an ensemble average execution time; useful for timing code with very short execution time| +|%who, %who_ls, %whos| Display variables defined in interactive namespace, with varying levels of information/verbosity| +|%xdel| variable Delete a variable and attempt to clear any references to the object in the IPython internals| + %matplotlib inline|Matplotlib Integration| +|%load_ext lab_black| Code formatting during save | +|%matplotlib widget| plot mathplot| +|%load_ext tensorboard| load tensorflow board| +|%matplotlib widget|Load Matplotlib | + \ No newline at end of file diff --git a/Python/Numpy/Indexing, Slicing and Iterating.md b/Python/Numpy/Indexing, Slicing and Iterating.md new file mode 100644 index 0000000..f9a8657 --- /dev/null +++ b/Python/Numpy/Indexing, Slicing and Iterating.md @@ -0,0 +1,37 @@ +--- +title: Indexing, Slicing and Iterating +updated: 2022-04-25 10:19:34Z +created: 2022-04-25 10:15:11Z +--- + +```python +a = np.array([[1,2], [3, 4], [5, 6]]) +``` +Output: +array([[1, 2], + [3, 4], + [5, 6]]) + +```python +# if we want to get multiple elements +# for example, 1, 4, and 6 and put them into +# a one-dimensional array we can enter the +# indices directly into an array function +np.array([a[0, 0], a[1, 1], a[2, 1]]) +``` +Output: +array([1, 4, 6]) + +```python +# we can also do that by using another form of array indexing, which essentiall "zips" the first list and the +# second list up +print(a[[0, 1, 2], [0, 1, 1]]) +``` +Output: +array([1, 4, 6]) + + +## Boolean Indexing +```python +print(a[a>5]) # like in Pandas +``` diff --git a/Python/Numpy/Invert Image.md b/Python/Numpy/Invert Image.md new file mode 100644 index 0000000..7c707ec --- /dev/null +++ b/Python/Numpy/Invert Image.md @@ -0,0 +1,26 @@ +--- +title: Invert Image +updated: 2022-04-25 10:13:25Z +created: 2022-04-25 10:11:12Z +--- + +```python +from PIL import Image +from IPython.display import display + +im = Image.open('image.tiff') +array=np.array(im) + +mask=np.full(array.shape,255) + +# Now let's subtract that from the modified array +modified_array=array-mask + +# And lets convert all of the negative values to positive values +modified_array=modified_array*-1 + +# And as a last step, let's tell numpy to set the value of the datatype correctly +modified_array=modified_array.astype(np.uint8) + +display(Image.fromarray(modified_array)) +``` diff --git a/Python/Numpy/Trying Numpy with Datasets.md b/Python/Numpy/Trying Numpy with Datasets.md new file mode 100644 index 0000000..56f3ac4 --- /dev/null +++ b/Python/Numpy/Trying Numpy with Datasets.md @@ -0,0 +1,38 @@ +--- +title: Trying Numpy with Datasets +updated: 2022-04-25 10:28:56Z +created: 2022-04-25 10:20:03Z +--- + +```python +wines = np.genfromtxt("datasets/winequality-red.csv", delimiter=";", skip_header=1) +``` + +### So all rows combined but only the first column from them would be +```python +print("one integer 0 for slicing: ", wines[:, 0]) +``` + +### But if we wanted the same values but wanted to preserve that they sit in their own rows we would write +```python +print("0 to 1 for slicing: \n", wines[:, 0:1]) +``` + +### What if we want several non-consecutive columns? We can place the indices of the columns that we want into an array and pass the array as the second argument. Here's an example +```python +wines[:, [0,2,4]] +``` + + +### We can specify data field names when using genfromtxt() to loads CSV data. Also, we can have numpy try and infer the type of a column by setting the dtype parameter to None +```python +graduate_admission = np.genfromtxt('datasets/Admission_Predict.csv', dtype=None, delimiter=',', skip_header=1, + names=('Serial No','GRE Score', 'TOEFL Score', 'University Rating', 'SOP', + 'LOR','CGPA','Research', 'Chance of Admit')) +``` + +### We can retrieve a column from the array using the column's name for example, let's get the CGPA column and only the first five values. +```python +graduate_admission['CGPA'][0:5] +``` + diff --git a/Python/Pandas/Datetime.md b/Python/Pandas/Datetime.md new file mode 100644 index 0000000..41e2df2 --- /dev/null +++ b/Python/Pandas/Datetime.md @@ -0,0 +1,44 @@ +--- +title: Datetime +updated: 2022-05-03 19:17:05Z +created: 2022-05-03 19:02:29Z +--- + +# Converting column to datetime dtype while loading file. + +## Create a date parser function +```python +d_parser = lambda x: pd.to_datetime(x) +df = pd.read_csv(file_name.csv, parse_dates=['date_column'], date_parser=d_parser) +``` + +## If date is not in parseable format, use +```python +pd.to_datetime.strptime(x, format) +``` +## Eg. format for '2017-03-13 04-PM' is '%Y-%M-%D %I-%p' +## Datetime Formatting Codes - http://bit.ly/python-dt-fmt + + +# convert the 'Date' column to datetime format +```python +df['Date_new']= pd.to_datetime(df['Date']) +``` + +```python +df['Dates'] = pd.to_datetime(df['Dates'], format='%y%m%d') + +df['Date'] = df['Date'].astype('datetime64[ns]') + +dtype = pd.SparseDtype(np.dtype('datetime64[ns]')) +series = pd.Series(df.date, dtype=dtype) +df['date']=np.array(series) +``` + +[Source1](https://www.codegrepper.com/code-examples/python/convert+date+column+to+datetime+pandas) +[Source2](https://dataindependent.com/pandas/pandas-to-datetime-string-to-date-pd-to_datetime/) + +```python +from datetime import date +df_min_gr[df_min_gr['Date'] > pd.Timestamp(2005, 2, 25)] +``` \ No newline at end of file diff --git a/Python/Pandas/GroupBy.md b/Python/Pandas/GroupBy.md new file mode 100644 index 0000000..3640ada --- /dev/null +++ b/Python/Pandas/GroupBy.md @@ -0,0 +1,30 @@ +--- +title: GroupBy +updated: 2022-04-26 12:15:16Z +created: 2022-04-26 11:30:24Z +--- + + ## three broad categories of data processing to happen during the apply step: + - Aggregation of group data +``` +df.groupby("cancellation_policy")['review_scores_value'].mean() +df.groupby("cancellation_policy").agg({"review_scores_value":(np.nanmean,np.nanstd),"reviews_per_month":np.nanmean}) +``` + - Transformation of group data + ``` + cols = ["cancellation_policy", "review_scores_value"] + transform_df = df[cols].groupby("cancellation_policy").transform(np.nanmean) + transform_df.rename( + {"review_scores_value": "mean_review_scores"}, axis="columns", inplace=True +) +df = df.merge(transform_df, left_index=True, right_index=True) + ``` + - Filtration of group data + ``` + df.groupby("cancellation_policy").filter( + lambda x: np.nanmean(x["review_scores_value"]) > 9.2 +) +``` + +[source](https://ipython.readthedocs.io/en/7.23.0/interactive/magics.html) +[source2](https://coderzcolumn.com/tutorials/python/list-of-useful-magic-commands-in-jupyter-notebook-lab) \ No newline at end of file diff --git a/Python/Pandas/Index.md b/Python/Pandas/Index.md new file mode 100644 index 0000000..ec65808 --- /dev/null +++ b/Python/Pandas/Index.md @@ -0,0 +1,15 @@ +--- +title: Index +updated: 2022-04-25 17:42:01Z +created: 2022-04-25 17:40:58Z +--- + +```python + +df = df.set_index('time') +df = df.sort_index() + + +df = df.reset_index() +df = df.set_index(['time', 'user']) +``` diff --git a/Python/Pandas/RE.md b/Python/Pandas/RE.md new file mode 100644 index 0000000..613b1c3 --- /dev/null +++ b/Python/Pandas/RE.md @@ -0,0 +1,27 @@ +--- +title: RE +updated: 2022-04-25 17:40:12Z +created: 2022-04-25 17:38:20Z +--- + +# So that looks pretty nice, other than the column names. But if we name the groups we get named columns out +```python +pattern="(?P<First>^[\w]*)(?:.* )(?P<Last>[\w]*$)" + +# Now call extract +names=df["President"].str.extract(pattern).head() + +# =============== +pattern="(^[\w]*)(?:.* )([\w]*$)" + +# Now the extract function is built into the str attribute of the Series object, so we can call it +# using Series.str.extract(pattern) +dd = df["President"].str.extract(pattern) +dd.rename(columns={0:'first',1:'last'}).head(4) + +# ========================== +df["Born"]=df["Born"].str.extract("([\w]{3} [\w]{1,2}, [\w]{4})") +df["Born"]=pd.to_datetime(df["Born"]) + + +``` diff --git a/Python/Pandas/Scales.md b/Python/Pandas/Scales.md new file mode 100644 index 0000000..8c6744c --- /dev/null +++ b/Python/Pandas/Scales.md @@ -0,0 +1,25 @@ +--- +title: Scales +updated: 2022-04-26 12:39:56Z +created: 2022-04-26 12:38:15Z +--- + +``` +my_categories=pd.CategoricalDtype(categories=['D', 'D+', 'C-', 'C', 'C+', 'B-', 'B', 'B+', 'A-', 'A', 'A+'], ordered=True) + +# then we can just pass this to the astype() function +grades=df["Grades"].astype(my_categories) +``` +Output: +excellent A+ +excellent A +excellent A- +good B+ +good B +Name: Grades, dtype: category +Categories (11, object): [D < D+ < C- < C ... B+ < A- < A < A+] + +Usage +``` +grades[grades>"C"] +``` \ No newline at end of file diff --git a/Python/Pandas/ShiftRowsCalc.md b/Python/Pandas/ShiftRowsCalc.md new file mode 100644 index 0000000..78b86dc --- /dev/null +++ b/Python/Pandas/ShiftRowsCalc.md @@ -0,0 +1,62 @@ +--- +title: ShiftRowsCalc +updated: 2022-04-17 21:34:50Z +created: 2022-04-17 21:33:59Z +--- + +```python +import pandas as pd + +df = pd.DataFrame( + { + "ts": [ + "2020-01-01 21:00", + "2020-01-01 22:00", + "2020-01-01 21:05", + "2020-01-01 22:06", + "2020-01-01 22:00", + "2020-01-01 22:07", + "2020-01-01 23:01", + "2020-01-01 23:09", + "2020-01-01 23:01", + "2020-01-01 23:09", + ], + "action": [ + "start", + "start", + "stop", + "stop", + "start", + "stop", + "start", + "stop", + "start", + "stop", + ], + "kol1": [10, 20, 30, 40, 50, 60, 70, 80, 90, 100], + "name": [ + "aap", + "noot", + "aap", + "noot", + "aap", + "aap", + "noot", + "noot", + "olifant", + "olifant", + ], + "kol2": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], + } +) +df["day"] = pd.to_datetime(df.ts.str.slice(0, 10), format="%Y-%m-%d") +df["ts"] = pd.to_datetime(df["ts"], format="%Y-%m-%d %H:%M") + +df1 = df[df["action"] == "start"] +df2 = df[df["action"] == "stop"] +df2.reset_index(inplace=True) +df1.reset_index(inplace=True) +df3 = df1.join(df2, lsuffix="_a") +df3["diff"] = (df3["ts"] - df3["ts_a"]) / pd.Timedelta(minutes=1) +df3.groupby(["day", "name"]).sum() +``` \ No newline at end of file diff --git a/Python/Pandas/Where.md b/Python/Pandas/Where.md new file mode 100644 index 0000000..af9b7de --- /dev/null +++ b/Python/Pandas/Where.md @@ -0,0 +1,21 @@ +--- +title: Where +updated: 2022-04-25 17:47:25Z +created: 2022-04-25 17:45:33Z +--- + +```python + +admit_mask=df['chance of admit'] > 0.7 +df.where(admit_mask) + +# ======================= + +df[df["gre score"]>320] + +# ======================== + +df['chance of admit'].gt(0.7) & df['chance of admit'].lt(0.9) +# or chained +df['chance of admit'].gt(0.7).lt(0.9) +``` \ No newline at end of file diff --git a/Python/Pandas/select rows and columns.md b/Python/Pandas/select rows and columns.md new file mode 100644 index 0000000..ed656d6 --- /dev/null +++ b/Python/Pandas/select rows and columns.md @@ -0,0 +1,13 @@ +--- +title: select rows and columns +updated: 2022-04-25 12:11:53Z +created: 2022-04-25 12:06:56Z +--- + +df.loc['school1'] # school1 is an index name +df.T.loc['Name'] # first transpose and then a column name +df.loc[:,['Name', 'Score']] # Name and Score are columns + +df.drop('school1',axis=0) # drop the rows with index name school1 +df.drop("Name", inplace=True, axis=1) # drop the column with name Name + diff --git a/Python/Pandas/split columns.md b/Python/Pandas/split columns.md new file mode 100644 index 0000000..c52535e --- /dev/null +++ b/Python/Pandas/split columns.md @@ -0,0 +1,45 @@ +--- +title: split columns +updated: 2022-04-09 15:56:28Z +created: 2022-04-09 15:55:40Z +--- + +```python +import pandas as pd +import re + +d = { + "kol1": ["iets", "nogiets", "nogietssss", "laatste"], + "info": [ + "aap:1234,olifant:pp12,giraffe:34/0", + "aap:6789,giraffe:36/0", + "aap:0089,olifant:pp25,giraffe:40/0", + "aap:0089,olifant:pp25,neushoorn:blabla", + ], + "test": ["a1", "b2", "c3", "d4"], +} + +d1 = { + "kol1": ["iets1", "nogiets1", "nogietssss1", "laatste1"], + "info": [ + "{'aap':'1234','olifant':'pp12','giraffe':'34/0'}", + "{'aap':'6789','giraffe':'36/0'}", + "{'aap':'1189','olifant':'pp25','giraffe':'40/0'}", + "{'aap':'0089','olifant':'pp25','neushoorn':'blabla'}", + ], + "test": ["a1", "b2", "c3", "d4"], +} + +df = pd.DataFrame(d) +df1 = pd.DataFrame(d1) + + +def qq(x): + return eval("{" + re.sub(r"([\w\/]+)", r'"\1"', x) + "}") + + +res = df.join(df["info"].apply(lambda x: pd.Series(qq(x)))).drop(["info"], axis=1) +res1 = df1.join(df1["info"].apply(lambda x: pd.Series(eval(x)))).drop(["info"], axis=1) + +pd.concat([res, res1]).reset_index(drop=True) +``` \ No newline at end of file diff --git a/Python/Pipenv_Pyenv.md b/Python/Pipenv_Pyenv.md new file mode 100644 index 0000000..79529c9 --- /dev/null +++ b/Python/Pipenv_Pyenv.md @@ -0,0 +1,124 @@ +--- +title: Pipenv_Pyenv +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Pyenv & Pipenv + +## Installing Pyenv + +```bash +git clone https://github.com/pyenv/pyenv.git ~/.pyenv +echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc +echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc +``` + +```bash +pyenv versions # list installed python versions +pyenv install --list # list all available versions for installation +pyenv install <python_version> # install a version +pyenv uninstall <python_version> +pyenv global <python_version> # set version globally +pyenv local <python_version> # set the file .python-version with version 3.8.2 +pyenv shell <python_version> # set PYENV_VERSION environment variable as current +pyenv virtualenv <python_version> <environment_name> # create environment + +pyenv local <environment_name> # activate enviroment +pyenv activate <environment_name> # the same as above + +pyenv deactivate +``` + +## Using pipenv + +### Install + +pip install --user --upgrade pipenv + +### Usage + +```bash +cd project_directory +pipenv --python <python_version> install ## install python in this dir + +``` + +### Pipfile + +This is where we will keep of application requirements rather than using requirements.txt + +```bash +pipenv install fastapi uvicorn +``` + +Install some packages and the Pipfile is created with the packages added to Pipfile. By default, Pipenv will always install the newest versions of packages. + +```text +[[source]] +name = "pypi" +url = "https://pypi.org/simple" +verify_ssl = true + +[dev-packages] + +[packages] +fastapi = "*" +uvicorn = "*" + +[requires] +python_version = "3.7" +``` +Other file created is Pipfile.lock and is super important because it does two things: + +- Provides good security by keeping a hash of each package installed. +- Pins the versions of all dependencies and sub-dependencies, giving you replicable environments. + + +To install development packages just pass the --dev argument. Like + +```bash +pipenv install --dev ptpython +pipenv update ## update the packages +``` + +By default, pipenv install will **ONLY** install the base packages that should live on production. This is what we want. Passing the --dev parameter will allow us to install base AND dev packages locally or during the CI process. + +```bash +pipenv shell # Launching subshell in virtual environment… +pip list # lists all install packages and version +pipenv check # check for vulnerabilities (Checking PEP 508 requirements) +``` + + +## cheatsheet + +```bash +pipenv shell # Activate +python --version # Check version of Python + +python # Check path +>>> import sys +>>> sys.executable +quit() + +pipenv install camelcase # Install a package +pipenv --rm # remove the virtual environment +pipenv lock -r # Check local packages +pipenv lock -r > requirements.txt +pipenv uninstall <package-name> # Uninstall a package +pipenv install nose --dev # Install a dev package +pipenv install -r ./requirements.txt # Install from requirements.txt +pipenv check # Check security vulnerabilities +pipenv graph # Check dependency graph +pipenv install --ignore-pipfile # Ignore pipfile +pipenv lock # Set lockfile - before deployment +pipenv run python script.py +exit Exiting the virtualenv +``` + +[Source 1](https://dev.to/writingcode/the-python-virtual-environment-with-pyenv-pipenv-3mlo) + +[Source 2](https://hackernoon.com/reaching-python-development-nirvana-bb5692adf30c) + +[github pipenv](https://github.com/pypa/pipenv) \ No newline at end of file diff --git a/Python/Pytorch/Input.md b/Python/Pytorch/Input.md new file mode 100644 index 0000000..0321bb6 --- /dev/null +++ b/Python/Pytorch/Input.md @@ -0,0 +1,81 @@ +--- +title: Input +updated: 2022-06-22 17:55:33Z +created: 2022-06-22 13:15:11Z +--- + +### Drop unwanted columns +``` +import pandas as pd + +df = pd.read_csv('data/energydata_complete.csv', sep=',') +df.drop(columns=['date'], inplace=True) +``` + +### test isnull values +``` +df.isnull().sum() +``` + +### Outliers +``` +outliers = {} +for i in range(data.shape[1]): + min_t = data[data.columns[i]].mean() \ + - (3 * data[data.columns[i]].std()) + max_t = data[data.columns[i]].mean() \ + + (3 * data[data.columns[i]].std()) + count = 0 + for j in data[data.columns[i]]: + if j < min_t or j > max_t: + count += 1 + percentage = count / data.shape[0] + outliers[data.columns[i]] = "%.3f" % percentage + +all_values = outliers.values() +max_value = max(all_values) +max_value +``` + +### Get numerical data columns +``` +cols = df.columns +num_cols = df._get_numeric_data().columns +print(num_cols) +set(cols) - set(num_cols) +``` + + +### Normalize input +``` +from sklearn.preprocessing import MinMaxScaler + +scaler = MinMaxScaler() + +X = df.iloc[:, 1:] +Y = df.iloc[:, 0] +X = pd.DataFrame(scaler.fit_transform(X), columns=X.columns) + +``` + +[source](https://towardsdatascience.com/data-normalization-with-pandas-and-scikit-learn-7c1cc6ed6475) + +### Split +``` +from sklearn.model_selection import train_test_split + +train_ratio = 0.60 +validation_ratio = 0.20 +test_ratio = 0.20 + +x_train, x_test, y_train, y_test = train_test_split(X, + Y, + test_size=1 - train_ratio) + +x_val, x_test, y_val, y_test = train_test_split( + x_test, y_test, test_size=test_ratio / (test_ratio + validation_ratio)) + +print(x_train.shape, y_train.shape) +print(x_val.shape, y_val.shape) +print(x_test.shape, y_test.shape) +``` diff --git a/Python/Pytorch/Installation & running.md b/Python/Pytorch/Installation & running.md new file mode 100644 index 0000000..7b82525 --- /dev/null +++ b/Python/Pytorch/Installation & running.md @@ -0,0 +1,41 @@ +--- +title: Installation & running +updated: 2021-11-08 11:13:17Z +created: 2021-11-01 13:05:19Z +latitude: 52.38660000 +longitude: 5.27820000 +altitude: 0.0000 +--- + +# Installation +01/11/2021 15:17 +```python +import torch + +torch.cuda.is_available() +torch.cuda.current_device() +torch.cuda.device_count() +torch.cuda.get_device_name(0) +``` + +```python +device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') +print('Using device:', device) +print() + +#Additional Info when using cuda +if device.type == 'cuda': + print(torch.cuda.get_device_name(0)) + print('Memory Usage:') + print('Allocated:', round(torch.cuda.memory_allocated(0)/1024**3,1), 'GB') + print('Cached: ', round(torch.cuda.memory_reserved(0)/1024**3,1), 'GB') +``` + +```bash +watch -n 2 nvidia-smi +``` + +### run docker container +``bash +docker run -it --rm --gpus all --ipc=host --privileged -p 8888:8888 -v $(pwd):/workspace/fastbook fastdotai/fastai:latest ./run_jupyter.sh +``` \ No newline at end of file diff --git a/Python/Pytorch/Summary Book Deep Learning for Coders with Fastai .md b/Python/Pytorch/Summary Book Deep Learning for Coders with Fastai .md new file mode 100644 index 0000000..5264c19 --- /dev/null +++ b/Python/Pytorch/Summary Book Deep Learning for Coders with Fastai .md @@ -0,0 +1,92 @@ +--- +title: Summary Book Deep Learning for Coders with Fastai and PyTorch +updated: 2021-11-08 14:15:52Z +created: 2021-11-08 11:13:26Z +latitude: 52.38660000 +longitude: 5.27820000 +altitude: 0.0000 +--- + +# Jargon + +- **architecture**: template of the model that we’re trying to fit; i.e., the actual mathematical function that we’re passing the input data and parameters to +- **parameters**: weights +- **independent variable**: data without labels +- **dependent variable**: label +- **label**: The data that we’re trying to predict, such as “dog” or “cat” +- **predictions**: result of the model +- **loss**: measure of performance. The entire purpose of **loss** is to define a “measure of performance” that the training system can use to update weights automatically +- **epochs**: an epoch is one complete pass through the dataset +- **hyperparameters**: parameters about parameters, since they are the higher-level choices that govern the meaning of the weight parameters. +- **CNN**: Convolutional neural network; a type of neural network that works particularly well for computer vision tasks +- **NLP**: natural language processing +- **segmentation** Creating a model that can recognize the content of every individual pixel in an image. + +![31156c4015ea6f9c5ae11c6a02d28cbe.png](../../_resources/31156c4015ea6f9c5ae11c6a02d28cbe.png) + +**Positive feedback look**: the more the model is used, the more biased the data +becomes, making the model even more biased, and so forth. + +```python +from fastai.vision.all import * +path = untar_data(URLs.PETS)/'images' + +def is_cat(x): return x[0].isupper() +dls = ImageDataLoaders.from_name_func( + path, get_image_files(path), valid_pct=0.2, seed=42, + label_func=is_cat, item_tfms=Resize(224)) + +learn = cnn_learner(dls, resnet34, metrics=error_rate, pretrained=True) +learn.fine_tune(epochs=1) +learn.show_results(max_n=6, figsize=(7,8)) +``` +- **Image**DataLoaders: First part of name indicate type of data eq image or text + - **Tabular**DataLoaders, **Text**DataLoaders or **Segmentation**DataLoaders +- **valid_pct**=0.2 size **validation set** (==developement set) used to **measure the accuracy of the model**. Remaining is the **training set** +- **seed**=42 random seed to thesame value every time we run this code, which means we **get the same validation set** every time we run it. + +fastai will always show you your **model’s accuracy** using only the **validation set** +The longer you train (with not enough data) for, the better your accuracy will get on the training set; the validation set accuracy will also improve for a while, but eventually it will start getting worse as the model starts to **memorize the training set** rather than **finding generalizable underlying patterns in the data**. When this happens, we say that the model is **overfitting**. +**test set** hold back from the training process, we must hold back the test set data even from ourselves. It cannot be used to improve the model; it can be used only to evaluate the model at the very end of our efforts. Test data is totally hidden. +Hiring third party, then use a good test set for mitigating risks + +![a8e058c2e06084485b3140b75a20e78e.png](../../_resources/a8e058c2e06084485b3140b75a20e78e.png) +*Many methods to avoid overfitting, but use them only when confirmed that overfitting is occurring.* + +- learn = **cnn_learner**(dls, resnet34, metrics=error_rate): convolutional neural network (CNN) + - dls: the data + - resnet34: 34 in resnet34 refers to the number of layers in the resnet + Models using architectures with more layers take longer to train and are more prone to overfitting, when using more data, they can be quite a bit more accurate. + - metric: measures the quality of the model’s predictions using the validation set, and will be printed at the end of each epoch. +![7e3dd277a7935550cabb842d0b3e91ef.png](../../_resources/7e3dd277a7935550cabb842d0b3e91ef.png) + - error_rate: what percentage of images in the validation set are being classified incorrectly + - accuracy = 1.0 - error_rate + - parameter **pretrained** defaults to True (*transfer learning*). When using a pretrained model, cnn_learner will remove the last layer and replace it with one or more new layers with randomized weights, of an appropriate size for the dataset you are working with. This last part of the model is known as the **head**. +- learn.**fine_tune**(epochs=1) + - adapt a pretrained model for a new dataset + - **fit** method will overwrite all weights of the pretrained model +- learn.**show_results**(max_n=6, figsize=(7,8)) + +Computer vision datasets are normally structured in such a way that the **label** for an image is part of the filename or path - most commonly the parent folder name. + +A **Transform** contains code that is applied automatically during training: +- **item_tfms**: applied to each item eq item is resized to a 224-pixel square +- **batch_tfms**: applied to a batch of items at a time using the GPU + +## upload a picture +```python +from fastbook import * # requiered for widgets +uploader = widgets.FileUpload() +uploader +``` + +## test +```python +img = PILImage.create(uploader.data[0]) +is_cat,_,probs = learn.predict(img) +print(f"Is this a cat?: {is_cat}.") +print(f"Probability it's a cat: {probs[1].item():.6f}") +``` + +**classification model**: attempts to predict a class, or category. That is, it’s predicting from a number of discrete possibilities, such as “dog” or “cat.” +**regression model** is one that attempts to predict one or more numeric quantities, such as a temperature or a location. Using y_range parameter to define the range of the targets diff --git a/Python/Virtulenv.md b/Python/Virtulenv.md new file mode 100644 index 0000000..a6bd517 --- /dev/null +++ b/Python/Virtulenv.md @@ -0,0 +1,40 @@ +--- +title: Virtulenv +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Virtualenv + +[latest doc](https://virtualenvwrapper.readthedocs.io/en/latest/command_ref.html) + + +#### remove virtual environment +rmvirtualenv name + +#### create virtual environment +mkvirtualenv -p python3.6 name + +#### switch virtual environmen +workon name + +#### List all of the environments. +lsvirtualenv + +#### Navigate into the directory of the currently activated virtual environment, +cdvirtualenv + +#### so you can browse its site-packages, for example. +cdsitepackages + +#### Like the above, but directly into site-packages directory. +lssitepackages +Shows contents of site-packages directory. + +* * * + +## Anaconda +#### Om python3.5 environment te creeeren +conda create -n py35 python=3.5 anaconda +source activate py35 +source deactivate diff --git a/Python/Visualisation/Mathplotlib Architecture.md b/Python/Visualisation/Mathplotlib Architecture.md new file mode 100644 index 0000000..15bbc3d --- /dev/null +++ b/Python/Visualisation/Mathplotlib Architecture.md @@ -0,0 +1,63 @@ +--- +title: Mathplotlib Architecture +updated: 2022-05-02 14:30:40Z +created: 2022-05-02 10:04:42Z +--- + + +![0b8629bbcc1c0bf173fcefc7ea92df53.png](../../_resources/0b8629bbcc1c0bf173fcefc7ea92df53.png) + + +### difference between procedural and declarative visualization libraries +- pyplot scripting layer is a procedural method for building a visualization in that we tell the underlying software which drawing actions we want it to take in order to render our data. +- declarative methods for visualizing data. HTML is a great example of this. The popular JavaScript library, for instance,D3.JS is an example of a declarative information visualization method. + +### Matlablib layers +- **backend**: + concrete implementations of the abstract interface classes + - FigureCanvas: a concrete implementation which knows how to insert itself into a native Qt window + - Renderer: provide a low-level drawing interface for putting ink onto the canvas. + - Event: signals the callback dispatcher to generate the events so upstream listeners can handle them +```python +import numpy as np +import matplotlib.pyplot as plt + +def on_press(event): + if event.inaxes is None: return + for line in event.inaxes.lines: + if event.key=='t': + visible = line.get_visible() + line.set_visible(not visible) + event.inaxes.figure.canvas.draw() + +fig, ax = plt.subplots(1) + +fig.canvas.mpl_connect('key_press_event', on_press) + +ax.plot(np.random.rand(2, 20)) + +plt.show() +``` +- **artist**: middle layer of the matplotlib stack, and is the place where much of the heavy lifting happens. + Artist is the object that knows how to take the Renderer and put ink on the canvas. Everything you see in a matplotlib Figure is an Artist instance, + The coupling between the Artist hierarchy and the backend happens in the draw method. The API works very well, especially for programmers, and is usually the appropriate programming paradigm when writing a web application server, a UI application, or perhaps a script to be shared with other developers. + Two types of Artist: + - **Primitive artists** represent the kinds of objects you see in a plot: Line2D, Rectangle, Circle, and Text. + - **Composite artists** are collections of Artists such as the Axis, Tick, Axes, and Figure. Each composite artist may contain other composite artists as well as primitive artists. The most important composite artist is the Axes. +- **scripting**. + - Most special-purpose languages for data analysis and visualization provide a lighter scripting interface to simplify common tasks, and matplotlib does so as well in its matplotlib.pyplot interface. + - pyplot is a stateful interface that handles much of the boilerplate for creating figures and axes and connecting them to the backend of your choice, and maintains module-level internal data structures representing the current figure and axes to which to direct plotting commands. + ``` + import matplotlib.pyplot as plt + ``` + - When the pyplot module is loaded, it parses a local configuration file in which the user states, among many other things, their preference for a default backend. + - The pyplot interface is a fairly thin wrapper around the core Artist API which tries to avoid as much code duplication as possible by exposing the API function, call signature and docstring in the scripting interface with a minimal amount of boilerplate code. + +### matplotlib spends a lot of time transforming coordinates from one system to another. These coordinate systems include: +- data: the original raw data values +- axes: the space defined by a particular axes rectangle +- figure: the space containing the entire figure +- display: the physical coordinates used in the output (e.g. points in PostScript, pixels in PNG) + +[source](https://www.aosabook.org/en/matplotlib.html) + diff --git a/Python/Visualisation/Matplotlib Code.md b/Python/Visualisation/Matplotlib Code.md new file mode 100644 index 0000000..85dd2dc --- /dev/null +++ b/Python/Visualisation/Matplotlib Code.md @@ -0,0 +1,11 @@ +--- +title: Matplotlib Code +updated: 2022-05-02 14:33:55Z +created: 2022-05-02 14:30:43Z +--- + +- pyplot is going to retrieve the current figure with the function **gcf** and then get the current axis with the function **gca**. +- pyplot just mirrors the API of the axis objects. So you can call the plot function against the pyplot module. But this is calling the axis plot functions underneath, so be aware. +- the function declaration from most of the functions in matplotlib end with an open set of keyword arguments. + + diff --git a/Python/Visualisation/Ten Simple Rules for Better Figures.md b/Python/Visualisation/Ten Simple Rules for Better Figures.md new file mode 100644 index 0000000..df189c0 --- /dev/null +++ b/Python/Visualisation/Ten Simple Rules for Better Figures.md @@ -0,0 +1,40 @@ +--- +title: Ten Simple Rules for Better Figures +updated: 2022-05-02 12:41:23Z +created: 2022-05-02 12:05:33Z +--- + +## Rule 1: Know Your Audience + it is important to identify, as early as possible in the design process, the audience and the message the visual is to convey. The graphical design of the visual should be informed by this intent. + +## Rule 2: Identify Your Message +A figure is meant to express an idea or introduce some facts or a result that would be too long (or nearly impossible) to explain only with words, be it for an article or during a time-limited oral presentation. In this context, it is important to clearly identify the role of the figure, i.e., what is the underlying message and how can a figure best express this message? Once clearly identified, this message will be a strong guide for the design of the figure. + +## Rule 3: Adapt the Figure to the Support Medium +A figure can be displayed on a variety of media, such as a poster, a computer monitor, a projection screen (as in an oral presentation), or a simple sheet of paper (as in a printed article). Each of these media represents different physical sizes for the figure, but more importantly, each of them also implies different ways of viewing and interacting with the figure. + +## Rule 4: Captions Are Not Optional +you cannot explain everything within the figure itself—a figure should be accompanied by a caption. The caption explains how to read the figure and provides additional precision for what cannot be graphically represented. + +## Rule 5: Do Not Trust the Defaults +Since settings are to be used for virtually any type of plot, they are not fine-tuned for a specific type of plot. In other words, they are good enough for any plot but they are best for none. All plots require at least some manual tuning of the different settings to better express the message, be it for making a precise plot more salient to a broad audience, or to choose the best color-map for the nature of the data. + +## Rule 6: Use Color Effectively +If you have no need for color, you need to ask yourself, “Is there any reason this plot is blue and not black?” If you don't know the answer, just keep it black. The same holds true for color-maps. Do not use the default color-map (e.g., jet or rainbow) unless there is an explicit reason to do so. +- Sequential: one variation of a unique color, used for quantitative data varying from low to high. +- Diverging: variation from one color to another, used to highlight deviation from a median value. +- Qualitative: rapid variation of colors, used mainly for discrete or categorical data. + +## Rule 7: Do Not Mislead the Reader +Using pie charts or 3-D charts to compare quantities. These two kinds of plots are known to induce an incorrect perception of quantities and it requires some expertise to use them properly. As a rule of thumb, make sure to always use the simplest type of plots that can convey your message and make sure to use labels, ticks, title, and the full range of values when relevant. + +## Rule 8: Avoid “Chartjunk” +Unnecessary or confusing visual elements found in a figure that do not improve the message (in the best case) or add confusion (in the worst case). (Edward Tutfe) +[Junk Charts](https://junkcharts.typepad.com/) + +## Rule 9: Message Trumps Beauty +most of the time, you may need to design a brand-new figure, because there is no standard way of describing your research. In such a case, browsing the scientific literature is a good starting point. If some article displays a stunning figure to introduce results similar to yours, you might want to try to adapt the figure for your own needs + +## Rule 10: Get the Right Tool + +[Source](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003833) \ No newline at end of file diff --git a/Python/Visualisation/Theory.md b/Python/Visualisation/Theory.md new file mode 100644 index 0000000..29c77bb --- /dev/null +++ b/Python/Visualisation/Theory.md @@ -0,0 +1,31 @@ +--- +title: Theory +updated: 2022-04-26 16:48:19Z +created: 2022-04-26 14:38:52Z +--- + +Alberto Cairo (Visualization Wheel) +[Alberto Cairo's Visualization Wheel](https://ryanwingate.com/visualization/guidelines/visualization-wheel/) + +### What's a heuristic? +A heuristic is a process or rule that is meant to guide you in decision making. Is by definition not known to be optimal or perfect, but to be practical in nature. Heuristics are meant to be followed until you've a reason to deviate from them. + +Edward Tufte (book: The Visual Display of Quantitative Information) +introduces two interesting **graphical heuristics**: +- the data-ink ratio +Tufte's first graphical heuristic is data-ink ratio. +Tufte defines data-ink as the non-erasable core of a graphic. In other words, the data-ink is essential to the sense-making process for a given variable. +Example +![95570cc80acedfbfa19a48f3f89b9024.png](../../_resources/95570cc80acedfbfa19a48f3f89b9024.png) +[More examples](https://www.darkhorseanalytics.com/) + +- chart junk +Tufte is much more damning of chart-junk than he is of other forms of non-data ink. +He suggests that artistic decorations on statistical graphs are like weeds in our data graphics. +There's really three kinds of chart-junk: + - **unintended optical art**. For instance, excessive shading or patterning of chart features + ![8bc6b50bc979e6571d513235dd90457d.png](../../_resources/8bc6b50bc979e6571d513235dd90457d.png) + These phenomenon are called moiré patterns. + - Tufte suggested the **grid is both unnecessary as data ink**, but also causes competition with the actual data being shared. Thinning, removing, or de-saturating grid lines makes it easier to see the data, instead of being overwhelmed by the number of lines on the page. **Direct labeling of data** is another great way to reduce this form of chart-junk. + - the duck. Broadly, Tufte is referring to non-data creative graphics, whether they be line art or photographs and they're inclusion in the chart. +As you go about creating your data science graphics, it's worth not only reflecting on the principles you use and the results you are sharing, but also the process by which you came up to create the graphics. \ No newline at end of file diff --git a/Python/Webserver.md b/Python/Webserver.md new file mode 100644 index 0000000..cd77650 --- /dev/null +++ b/Python/Webserver.md @@ -0,0 +1,11 @@ +--- +title: Webserver +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +Webserver + +```python +python -m http.server +``` diff --git a/Python/XLS.md b/Python/XLS.md new file mode 100644 index 0000000..4e4959b --- /dev/null +++ b/Python/XLS.md @@ -0,0 +1,16 @@ +--- +title: XLS +updated: 2022-01-17 20:43:48Z +created: 2022-01-17 20:43:09Z +latitude: 52.38660000 +longitude: 5.27820000 +altitude: 0.0000 +--- + +```bash +pip install openpyxl +``` + +```python +employment = pd.read_excel("data/unemployment.xlsx") +``` \ No newline at end of file diff --git a/Python/YAML.md b/Python/YAML.md new file mode 100644 index 0000000..230b076 --- /dev/null +++ b/Python/YAML.md @@ -0,0 +1,30 @@ +--- +title: YAML +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +YAML + +```python +# -*- coding: utf-8 -*- +import yaml +import io + +# Define data +data = {'a list': [1, 42, 3.141, 1337, 'help', u'€'], + 'a string': 'bla', + 'another dict': {'foo': 'bar', + 'key': 'value', + 'the answer': 42}} + +# Write YAML file +with io.open('data.yaml', 'w', encoding='utf8') as outfile: + yaml.dump(data, outfile, default_flow_style=False, allow_unicode=True) + +# Read YAML file +with open("data.yaml", 'r') as stream: + data_loaded = yaml.safe_load(stream) + +print(data == data_loaded) +``` \ No newline at end of file diff --git a/Python/pip.md b/Python/pip.md new file mode 100644 index 0000000..cdba927 --- /dev/null +++ b/Python/pip.md @@ -0,0 +1,86 @@ +--- +title: pip +updated: 2021-09-01 19:18:20Z +created: 2021-05-04 14:58:11Z +--- + +# Python pip commands + +[pip install documentation](https://pip.pypa.io/en/stable/reference/pip_install/#id18) + +## upgrade pip. watch -m parameter "run module as an module" + +```bash +python3 -m pip install -U pip +``` + +```bash +pip uninstall < package-name > +``` + +[pypi](https://pypi.org/) + +```bash +pip search <package you want to search for> // also multiple packages +``` + +```bash +pip install < package-name > +``` + +```bash +pip show < package-name > +``` + +### update all packages + +```bash +sudo pip install -U $(pip freeze | awk '{split($0, a, "=="); print a[1]}') +``` + +### install specific (not latest version) + +```bash +pip install <package-name>==version +``` + +### latest version + +```bash +pip list --outdated --format=columns +pip-check +``` + +### dump current packages, with version number + +```bash +pip freeze > requirements.txt +``` + +### install or upgrade + +```bash +pip install -r requirements.txt +pip install --upgrade -r requirements.txt +``` + +### requirements.txt contains ie: + +```bash +Flask==0.12 +click==4.1 +requests==2.13.0 +``` + +### logical operators can be used + +```bash +certifi>=2018.11.29 +chardet>=3.0.4 +``` + +### new versions of packages are not always backwards compatible!!!!!!! + +```bash +pip list | grep bleach | xargs sudo pip uninstall -y +``` diff --git a/SE/Data/Data Management at Scale.md b/SE/Data/Data Management at Scale.md new file mode 100644 index 0000000..34ca025 --- /dev/null +++ b/SE/Data/Data Management at Scale.md @@ -0,0 +1,56 @@ +--- +title: Data Management at Scale +updated: 2022-05-21 12:13:15Z +created: 2022-05-21 12:13:02Z +--- + +# Data Management at Scale + +## Chapter 1 + +[Datafication: is the transformation of social action into online quantified data, thus allowing for real-time tracking and predictive analysis](https://www.datasciencecentral.com/the-concept-of-datafication-definition-amp-examples/) + +data harmonization: bringing amounts of data into a particular context + +Data Management: [DAMA](https://www.dama.org/cpages/body-of-knowledge) + +[Data Monetization:](https://www.gartner.com/en/information-technology/glossary/data-monetization) Data Monetization refers to the process of using data to obtain quantifiable economic benefit. Internal or indirect methods include using data to make measurable business performance improvements and inform decisions. External or direct methods include data sharing to gain beneficial terms or conditions from business partners, information bartering, selling data outright (via a data broker or independently), or offering information products and services (for example, including information as a value-added component of an existing offering). + +Data Proliferation: same data gets distributed across many applications and +database + +Dataintensiveness:read-versus-write ratio is changing significantly. +Optimize for read: dupklication of data and/or applications optimized for read. + +Devops and smaller applications (microservices, k8s, domain-driven design, serverless computing) result in increased complexity and increased demand to beter control data. + +[DataOps](https://dataopsmanifesto.org/en/) +Focus on data interoperability, +the capture of immutable events, and reproducible and loose coupling. + +Bring the data to the application vs dont move the data becomes less important op cloud. Important for SaaS and Machine Learning as a Service (MLaaS) But will fragment the data possible. + +Insights about where data origina‐ +ted and how data is distributed are crucial. A stronger internal governance is required. The trend of stronger control is contrary to the methodologies for fast software development, which involves less documentation and fewer internal controls. + +For advanced analytics, such as machine learning, leaving context out can be a big problem because if the data is meaningless, it is impossible to correctly predict the future. + +[IntegrationDatabase: is a database which acts as the data store for multiple applications, and thus integrates data across these applications (in contrast to an ApplicationDatabase).](https://martinfowler.com/bliki/IntegrationDatabase.html) + +The “big ball of mud” describes a system architecture that is monolithic, difficult to understand, hard to maintain, and tightly coupled because of its many dependencies. Data warehouses, with their layers, views, countless tables, relationships, scripts, ETL jobs, and scheduling flows, often result in a chaotic web of dependencies. The lack of agility often becomes a concern => risk development of work-arounds => technical debt. + +Data lakes (multiple formats: structured, semi-structured, and unstructured),just like data warehouses, are considered centralized (monolithic) data +repositories, but they differ from warehouses because they store data before it has +been transformed, cleansed, and structured. + +Data warehouses are usually engineered with RDBMs, while data lakes are commonly engineered with distributed databases or NoSQL systems or public cloud. +Dumping in raw application structures—exact copies—is fast and allows data analysts and scientists quick access. However, the complexity with raw data is that use cases always require reworking the data. Data quality problems have to be sorted out, aggregations are required, and enrichments with other data are needed to bring the data into context. This introduces a lot of repeatable work and is another reason why data lakes are typically combined with data warehouses. Data warehouses, in this combination, act like high-quality repositories of cleansed and harmonized data, while data lakes act like (ad hoc) analytical environments, holding a large variety of raw data to facilitate analytics. +Data-lake-implementation failure rate of more than 60%. Data lake implementations typically fail, in part, because of their immense complexity, difficult maintenance, and shared dependencies. Other reasons include management resistance, internal politics, lack of expertise, and security and governance challenges + +**Scaled Architecture**: a reference and domain-based architecture with a set of blueprints, designs, principles,models, and best practices that simplifies and integrates data management across the entire organization in a distributed fashion. + +domain-agnostic: the topic is explained without taking examples of any specific domain. + +## Chapter 2 Introducing the Scaled Architecture: Organizing Data at Scale +### How can you distribute data efficiently while retaining agility, security, and control? + diff --git a/SE/Data/Definitions.md b/SE/Data/Definitions.md new file mode 100644 index 0000000..eabc904 --- /dev/null +++ b/SE/Data/Definitions.md @@ -0,0 +1,10 @@ +--- +title: Definitions +updated: 2022-05-21 13:16:46Z +created: 2022-05-21 12:20:35Z +--- + +Agnostic (data): +- In computing, a device or software program is said to be agnostic or data agnostic if the method or format of data transmission is irrelevant to the device or program’s function. This means that the device or program can receive data in multiple formats or from multiple sources, and still process that data effectively. [source](https://en.wikipedia.org/wiki/Agnostic_(data)) +- Back-pressure: if downstream is not keeping up with the flow of data the way slowdown the data-flow => take care sources produce less data or pause for a little while. +- control latency vs throughput: process a specific message as fast as possible and get it to it's destination. On the other side process all data as fast as possible, so throughput is as much as possible so sacrifice a few seconds latency \ No newline at end of file diff --git a/SE/Git/Default remote and branch.md b/SE/Git/Default remote and branch.md new file mode 100644 index 0000000..37180bd --- /dev/null +++ b/SE/Git/Default remote and branch.md @@ -0,0 +1,33 @@ +--- +title: Default remote and branch +updated: 2021-05-05 12:32:22Z +created: 2021-05-04 14:58:11Z +--- + +# GIT + +## Default remote and branch + +```bash +cat .git/config // wil list the remote and branch +git add . // adds all modifications to the staging area +git commit -m "some text" +git push // push all modifications to the upstream repository +``` + +### make initial ssh key: +### register the rsa.public key on remote host +```bash +ssh-keygen -t rsa +``` + +### add global username and email address +```bash +git config --global user.name "Your Name" +git config --global user.email "youremail@yourdomain.com" +``` +### Test connection + +- ssh -T [git@github.com](mailto:git@github.com) +- ssh -T [git@bitbucket.org](mailto:git@bitbucket.org) +enter a phrase password \ No newline at end of file diff --git a/SE/Git/New Repo.md b/SE/Git/New Repo.md new file mode 100644 index 0000000..6db25a7 --- /dev/null +++ b/SE/Git/New Repo.md @@ -0,0 +1,43 @@ +--- +title: New Repo +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# New Repo + +## New repository when local reposity is created first + +**remote:** +create remote repository in regular way, with *remoteRepoName* + +**local:** + +```bash +git init +git add . +git commit -m 'Init version' +git remote add origin <ssh://git@bitbucket.org/gitbucketwillems59/**remoteRepoName**.git> +git pull origin master --allow-unrelated-histories // merge remote with local +git push origin master // simply push to repository +``` + +--- + +- Edit .gitignore to match the file you want to ignore +- git rm --cached /path/to/file +- [sample ignore files for languages](https://github.com/github/gitignore) + +## log + +```bash +alias glog="git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit" + +# usage: +glog -n 5 +``` + + +## Setup git + +[git book])https://www.git-scm.com/book/en/v2/Customizing-Git-Git-Configuration) \ No newline at end of file diff --git a/SE/OpenSource/Why open source_.md b/SE/OpenSource/Why open source_.md new file mode 100644 index 0000000..60114ac --- /dev/null +++ b/SE/OpenSource/Why open source_.md @@ -0,0 +1,33 @@ +--- +title: Why open source_ +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Why open source + +Cloud computing brought a completely new and agile way of thinking about the vendor/consumer business model with the pay-as-you-grow model and SaaS. This model forced vendors to become more efficient and accurate with respect to how they deliver their software and think about the value behind their product. +The world where the consumer was willing to pay for “shelfware” is long gone, as **cloud** has also empowered the consumer to be more sophisticated with the way they measure the value behind a product and make data-driven decisions. + +With **open source**, the picture is a bit more complex. So, what is the value of being an open source product if the cost is no longer the key driving force? +Open source changes the relationship between the vendor and the consumer from a relationship of mistrust and that of a passive consumer of a veritable “black box,” to a relationship of trust and partnership. Open source makes it possible for consumers to know exactly what they are getting before they buy the product, along with more ways to control the way they will consume a given product or shape it through code contribution, feature requests, etc. With non-open source technology, a consumer is dependent entirely on the vendor every step of the way. +The “free” part in open source also allows consumers to choose when to pay for a given product based on their adoption stage, and value that they are getting from the product. Many open source companies will allow you to use their products completely free with the assumption that as you grow and get more substantial value from the product, you’ll need support and specific hardening, and this is where they will see monetization. So, in a nutshell, open source allows consumers to pay for products only when they feel that they are getting enough value from them or want to grow with the value. + +Open source is not just a technology choice but plays a key role from a business value perspective: scale. Successful open source companies tend to scale much faster than closed source alternatives. +The trust relationship that open source companies often have with their consumer also creates a higher degree of loyalty, and thus stickiness, which is one of the key factors that leads to that difference in scale when compared with non-open source alternatives. + +The monetization process starts after the customer is already convinced and has been using the product for a while. Therefore, they would be much more receptive to moving forward quickly. + +When Amazon started to provide managed offering of specific open source products, they were breaking right into the middle of this assumption. That broke the entire “trust system” that stood behind many +of the open source models. + +___Amazon are abusing open source products to their benefit___ + +Google was the first to realize this with the launch of Kubernetes as an open source project and win more cloud business by demonstrating a “non lock-in” strategy with open source. +what’s in it for Google? t’s the same strategy that helped Google beat Apple in the smartphone business — launching Android as an open alternative to iOS. +Taking the open source route is an effective strategy to win the hearts and minds of developers. + +[Source](https://thenewstack.io/the-amazon-effect-on-open-source/) + +Other sources: +https://opensource.com/life/15/12/why-open-source \ No newline at end of file diff --git a/SE/SWArchitecture/Tools.md b/SE/SWArchitecture/Tools.md new file mode 100644 index 0000000..aa3bc7c --- /dev/null +++ b/SE/SWArchitecture/Tools.md @@ -0,0 +1,13 @@ +--- +title: Tools +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Url's to source code metric tooling + +[Werk niet meer: XRay](http://xray.inf.usi.ch/xray.php) + +[Codecity: werkt](http://wettel.github.io/codecity.html) + +[Alleen windows: Source monitor](http://www.campwoodsw.com/sourcemonitor.html) diff --git a/SE/Security/Digital_Identity_terms.md b/SE/Security/Digital_Identity_terms.md new file mode 100644 index 0000000..da14ce5 --- /dev/null +++ b/SE/Security/Digital_Identity_terms.md @@ -0,0 +1,49 @@ +--- +title: Digital_Identity_terms +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Digital Identity Terms + +## Digital Identity + +‘Digital Identity’ is een verzameling kenmerken die betrekking hebben op een persoon. Een persoon kan meerdere digitale identiteiten voor dezelfde of verschillende diensten en voorzieningen hebben. De oude wijsheid geldt nog steeds: alles wat je publiceert op het web, of wat iemand anders over je heeft gepubliceerd, zal voor altijd daar staan. Zelfs reeds overleden mensen hebben digitale identiteiten. + +## Authenticatie + +"Bent u wie u zegt dat u bent?" Authenticatie beantwoordt de vraag: Bent u wie u zegt dat u bent? Een digitale service of systeem heeft informatie nodig over de gebruiker zodat deze toegang kan verkrijgen. Het herkennen van een persoon in de digitale omgeving kan op vele niveaus plaatsvinden. Dit varieert van een eenvoudig verzoek om een wachtwoord en een gebruikersnaam tot sterkere verificatie. + +## Multi-factor authenticatie (MFA) + +Wanneer u authenticatie naar een hoger niveau wilt brengen, kunt verschillende vormen van verificatie achtereenvolgens toepassen, bijvoorbeeld een wachtwoord in combinatie met een geautomatiseerd SMS-bericht of een eenmalige code in een authenticator app. Het gebruik van verschillende verificatiemethoden voor één gebruiker maakt identificatie betrouwbaarder en daardoor is een account minder makkelijk te misbruiken. Volgens Microsoft beperkt MFA het risico op misbruik van accounts met 99,9%! + +## Biometrische authenticatie + +Deze verificatiemethode was lange tijd alleen bekend in Sci-Fi en actiefilms. Tegenwoordig is het dagelijks gebruik om in te loggen op uw telefoon met behulp van een vingerafdruk of het tonen van uw gezicht op de camera. Het is moeilijk een biologische verificatiemethode te doorbreken. Ze worden vrij veilig beschouwd,dus sterk. Alhoewel we verwachten dat door toepassing van Deep-Fake technieken een camera niet meer veel toevoegt. Het nieuwste type biometrische authenticatie is gebaseerd op menselijk gedrag, zoals de manier van typen of hoe iemand loopt. + +## Autorisatie + +Wanneer iemand geautoriseerd is, krijgt die persoon het recht om iets te doen op basis van de taken die iemand moet verrichten. Sommige autorisaties worden automatisch verstrekt bij in diensttreding, andere worden door leidinggevenden verstrekt. En natuurlijk weer ingetrokken bij vertrek of verandering van werkplek. Vanuit de optiek van security en privacybescherming is het uitgangspunt om zo min mogelijk autorisaties te hebben. + +## IAM + +Identity & Access Management (IAM - identiteits- en toegangsmanagement) is binnen een organisatie een verzameling processen om medewerker- en/of klantgegevens ten behoeve van van identificatie en autorisatie geautomatiseerd te verwerken. Hoe groter de organisatie, hoe nuttiger de inzet van geautomatiseerde IAM-oplossingen. Al zien we meer en meer de ontwikkeling dat door torepassing van Federatie afscheid genomen wordt van traditioneel IAM. + +## Single Sign-on (SSO) + +Wanneer een systeem SSO biedt, hoeft een gebruiker zich slechts één keer aan te melden om toegang te verkrijgen tot gekoppelde applicaties of services. SSO vereenvoudigt de gebruikerservaring en werkstromen, maar levert ook hogere beveiliging omdat accounts voor losse applicaties niet meer zomaar gedeeld kunnen worden. + +## Privileged Access Management (PAM) + +Niet alle accounts zijn persoonsgebonden accounts. Sommige accounts zijn onpersoonlijk, zoals een admin-account, of een root-account. Deze accounts zijn geprivilegieerd, ze hebben bijzondere autorisaties en zijn dan ook vaak het doelwit van criminelen. Ze moeten op een extra beveiligde manier beheerd worden. Een PAM-systeem zorgt voor het op een veilige manier gebruiken van deze accounts. + +## Federatieve toegang + +IAM is een kostbaar proces, vandaar dat er steeds meer sprake is van federatieve toegang, waarbij bestaande digitale identiteiten worden hergebruikt. Als een al bestaande digitale identiteit van iemand vertrouwd kan worden, dan kan die persoon die identiteit ook gebruiken om (binnen bepaalde voorwaarden) toegang te krijgen. Dat maakt het voor zowel die persoon makkelijk (dat scheelt weer een account) als voor de organisatie makkelijker, want die persoonsgegevens hoeven niet beheerd te worden. Federatie maakt IAM schaalbaar. + +## Access Governance + +“Wie mag wat en waarom is dat zo?” Deze vraag is niet zomaar een vraag, het is de centrale vraagstelling rondom toegang. In de kern is IAM niet maar zomaar een technisch ICT probleem is, maar het is een vraagstuk van de hele bedrijfsvoering. IAM moet dan ook in de volle breedte worden beschouwd, zowel ten aanzien van medewerkers, als partners en klanten, als systeemaccount en zelfs dingen. Wij willen u graag helpen dit inzichtelijk te maken en de problemen en knelpunten met u op te lossen. + +[source Computable](https://www.computable.nl/artikel/advertorial/sponsored/6829152/5740344/10-digital-identity-termen-die-u-zou-moeten-beheersen.html) diff --git a/SE/Security/IAM.md b/SE/Security/IAM.md new file mode 100644 index 0000000..003deb7 --- /dev/null +++ b/SE/Security/IAM.md @@ -0,0 +1,62 @@ +--- +title: IAM +updated: 2022-01-09 15:17:28Z +created: 2022-01-09 13:20:59Z +latitude: 52.38660000 +longitude: 5.27820000 +altitude: 0.0000 +--- + +# Identity & Access Management (IAM) + +Everything that helps in managing identities and their access. It is about the “who” and the “what”: Who can do what in IT? Who can log on? Who can access which data? + +## Identity Management (abbr. of IAM) + +Include partners (B2P), customers and consumers (B2C), as well as non-human entities that have identities such as services (software processes) and internet-connected devices that make up the Internet of Things (IoT) +The challenge is for organizations: + +- manage all these identities to meet security and privacy requirements +- enabling business growth +- frictionless consumer/customer interaction +- personalized or tailored services and content. + Therefor appropriate strategy and loosely coupled, extensible and service-orientated IT architecture in place to enable a smooth transition to the **as-a-service model**, both in terms of service consumption (to reduce costs and boost productivity) and service provision (to add new revenue streams and improve consumer/customer engagement). + +*Essentially, the success of Digital Transformation depends on an ability to manage the access of everyone and everything to every digital service*. + +Organizations should plan to support **all kinds of identities** and ensure they have the tools to understand the level of assurance provided by each identity type so they can make informed decisions on how those identities can be used for specific transactions or interactions using risk-based scoring, and adaptive authentication and authorization systems. +Agile and flexible by separating identity and applications, and providing the backend systems required to make all the necessary connections using Application Program Interfaces (APIs) that bridge services, microservices and containers +These changes will result in a converged digital identity backend or **“Identity Fabric”** that can deliver as a utility all the identity services (including security and privacy) required by the growing number of new digital services enabled by digital transformation that will actively consume identity services. + +![bfd9c881c6664a63e2334228e4e57f83.png](../../_resources/bfd9c881c6664a63e2334228e4e57f83.png) +Also directory services as sort of a database of all the users, and Identity Verification for the first-time proof of an identity during the on-boarding process, e.g., + +## IAM consists of three core technologies (Traditional): + +- **User Lifecycle Management & Access Governance** ( ==\> **IGA (Identity Governance & Administration)**) + - **User Lifecycle Management** is also called **Identity Provisioning**. + - The entire identity management process from creating a digital identity when on-boarding a person such as a new employee, to retiring that digital identity. **Join, Move, Leave (JML)** + - The other part of User Lifecycle Management & Access Governance is about **requesting, approving, and reviewing** access entitlements. (who should have access to which systems, applications, and data) + - **Access reviews** are required for regulatory compliance, to check regularly whether the access is still needed or must be revoked. + - User Lifecycle Management is tightly coupled with **Identity Provisioning**: technically connecting IGA with the target systems such as Microsoft Active Directory or SAP, for e.g., creating user accounts, and assigning entitlements. + - <img src="../../_resources/19949739424c8962762275254b28a831.png" alt="19949739424c8962762275254b28a831.png" width="761" height="700" class="jop-noMdConv"> +- Access Management & Federation + - Access to systems at runtime. Requires authentication, i.e. the verification that credentials such as user name and password have been entered correctly. Federation to other systems, a concept where for instance a user is authenticated in his organization and then logs on to the organization’s tenant of a SaaS (Software as a Service) application such as Salesforce. + - **Access federation** is based on established standards such as OAuth2, SAML (Security Assertion Markup Language), and OIDC (OpenID Connect). The norm is SCIM. + - ![90294faf40d4ca7b98565abeec1db421.png](../../_resources/90294faf40d4ca7b98565abeec1db421.png) +- **Privileged Access Management PAM** ==> Administrators + - most technical discipline of IAM that deals with the specifics of highly privileged users such as administrators logging into systems, e.g., as Windows administrator or root user on Linux. + - Some of the aspects covered by PAM solutions are rotating passwords for shared accounts, and session management. + - Shared accounts such as root can be used by multiple persons. + - To mitigate the risks, users of privileged accounts must not use the same password. + - PAM ensures (amongst many other features) that privileged account users always get a new password for each new access. + - The session management part of PAM is about recording sessions, e.g., for forensics, and for logging and monitoring what happens during privileged access. + +## Con of Traditional IAM + +**Creating user accounts** in a range of systems such as Microsoft Active Directory, Microsoft Azure Active Directory, LDAP servers, on email servers, mainframes, business applications such as SAP, specialized banking apps, new SaaS services, and many more ***requires technical integration*** with these systems via connectors. That part of integration is **challenging**, despite **SCIM** (System for Cross-domain Identity Management) becoming increasingly established as a standard. + +## New trend towards IDaas (Identity as a Service) +- deployed using as-a-service models, avoiding the complex installation and operation of IAM on-premises. +- integration with legacy backends such as mainframes still requires setting up connectors. +- \ No newline at end of file diff --git a/SE/Theory/BoundedContext.md b/SE/Theory/BoundedContext.md new file mode 100644 index 0000000..0b62470 --- /dev/null +++ b/SE/Theory/BoundedContext.md @@ -0,0 +1,40 @@ +--- +title: BoundedContext +updated: 2022-04-03 09:23:29Z +created: 2021-05-04 14:58:11Z +--- + +# Bounded context (BC) + **== autonomous components (own domain models an ubiquitous language.** +* No dependencies on each other at runtime +* should be able to run in isolation +* updated/modified independent from each other + +When **CQRS** pattern (= Command Query Responsibility Segregation) in bounded context: +use events +* BC can respond to incoming events +* can publish outgoing events to subscribers + +(evens == one-way, asynchronous messages publish information about something that has already happened) + +Events enable to maintain loose coupling between BC + +Documentation should cover: which BC publishes which integration events, which BC subscribes to which integration events + +When BC is modified/updated events can change. A BC must be robust for such modifications ===> +## solution is a anti-corruption layer +* for verifying is an incoming integration event makes sense. ie type correct. +* translate incoming integration events (mapping to different type, converting to different version of event) + +Common approaches for getting data from legacy systems to a BC: +1. reading the database: BC that implements CQRS reads from the database of the legacy system (LC). Useful if LC has no API, but is now coupled!!!) +1. Generating events from database: implement monitoring program that is triggered when database is updated and generates events + decoupled BC, but an additional program to maintain +1. Modifying legacy system: publish events directly => potential consistency problem: ensure events are realy generated, when transactions are successful. + access to source code LC +1. Implications for event sourcing: if all events published by aggregates in that domain are persisted to event store. Where are the events stored? Should be ie able to rebuild a state after a failure. + +Source: komt nog + + + diff --git a/SE/Theory/Graphs.md b/SE/Theory/Graphs.md new file mode 100644 index 0000000..a879ed5 --- /dev/null +++ b/SE/Theory/Graphs.md @@ -0,0 +1,27 @@ +--- +title: Graphs +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# Graphs + +## Definition of a graph + +graph is pair of G=(V,E) +- v is a set of elements called vertices or nodes +- ie V = {A,B,C} +- E is set pairs (v<sub>j</sub>,v<sub>j</sub>) where v<sub>i</sub> $$\in V and v<sub>j</sub> $$\in V +- E = {(A,B), (A,C), (B,C)} + + +Diffent networks: +- Random +- small-world +- scale-free +- natural + +- Tree: undirected graph and has no cycles +- Rooted Tree: directed and designated root node. All edges pooint away or towards the root; when away is called out-tree; otherwise in-tree +- + \ No newline at end of file diff --git a/SE/Theory/ModelDrivenSoftwareDevelopment.md b/SE/Theory/ModelDrivenSoftwareDevelopment.md new file mode 100644 index 0000000..92075d2 --- /dev/null +++ b/SE/Theory/ModelDrivenSoftwareDevelopment.md @@ -0,0 +1,26 @@ +--- +title: ModelDrivenSoftwareDevelopment +updated: 2021-05-04 14:58:11Z +created: 2021-05-04 14:58:11Z +--- + +# 5 types: + + +1. Programming, just coding +2. Code Generation + automated programming ie what a IDE does. +3. Model Driven Abstraction (MDA) + Construct platform independent model that is transformed into an platform dependent model using [model transformations](http://www.theenterprisearchitect.eu/blog/2008/02/18/mda-and-model-transformation). From the platform dependent model the source code is generated. + The MDA is mainly aimed at technical variability. +4. Domain Specific Languages (DSL) + DSLs are small languages tailored to a specific domain. So multiple small models. + DSLs can vary in structure, variablity and notation. + Most DSL approaches contain a direct translation between DSLs and their execution by using code generation or the direct interpretation of the DSLs +5. Model Driven Engineering (MDE) => business-centric + We can also create models of the using system, i.e. creating models of the organization which is going to use the software. Once we have modeled the so-called problem domain, we can make translations to a model of the solution domain. These translations or transformations don’t necessarily have to be automated, in some cases that’s not even possible. However, they need to be supported by appropriate tools and methodologies. + [Horizontal transformations](http://www.theenterprisearchitect.eu/blog/2008/11/27/the-place-of-architecture-in-model-driven-engineering): from problem to solution domain + Vertical transformations: from solution domain to executable model. + The challenges of MDE are in appropriate tooling, methodologies, patterns, templates, reference architectures, and industry standards. + +[Source](http://www.theenterprisearchitect.eu/blog/2009/03/31/5-types-of-model-driven-software-development/) \ No newline at end of file diff --git a/SE/Theory/PrinciplesOfDatabases.md b/SE/Theory/PrinciplesOfDatabases.md new file mode 100644 index 0000000..604b90d --- /dev/null +++ b/SE/Theory/PrinciplesOfDatabases.md @@ -0,0 +1,20 @@ +--- +title: PrinciplesOfDatabases +updated: 2022-04-27 17:44:58Z +created: 2021-05-04 14:58:11Z +--- + +A **database** can be defined as a collection of related data items within a specific business process or problem setting + +A **database management system** (DBMS) is the software package used to define, create, use, and maintain a database. It typically consists of several software modules, each with their own functionality + +The combination of a DBMS and a database is then often called a **database system** + +The **database model or database schema** provides the description of the database data at different levels of detail and specifies the various data items, their characteristics, and relationships, constraints, storage details, etc. The database model is specified during database design and is not expected to change frequently. It is stored in the catalog + +The **database state** then represents the data in the database at a particular moment + + +[CAP Theorem](https://www.ibm.com/cloud/learn/cap-theorem) + +[Docker: how to run PostgreSQL and pgAdmin using Docker?](https://medium.com/quick-code/how-to-run-postgresql-and-pgadmin-using-docker-90638fde8bf) \ No newline at end of file diff --git a/SE/Theory/SOLID_Principles.md b/SE/Theory/SOLID_Principles.md new file mode 100644 index 0000000..bd45401 --- /dev/null +++ b/SE/Theory/SOLID_Principles.md @@ -0,0 +1,63 @@ +--- +title: SOLID_Principles +updated: 2022-04-03 09:29:17Z +created: 2021-05-04 14:58:11Z +--- + +# Five SOLID Principles + +- **S**ingle responsibility principle +- **O**pen closed principle +- **L**iskov substitution principle +- **I**nterface segregation principle +- **D**ependency Inversion Principle + +## Benefits + +- loosely coupled code = improves testability, maintainability, replace-ability +- cohesive +- encapsulating + + +## Single Responsibility Pronciple + +- A class should have one and only one reason to change, meaning that a class should have only one job. +- A God object is an object that knows too much or does too much :thumbsdown: The God object is an example of an anti-pattern. +- Too much responsibility leads to coupling (complicates change) +- Components with low cohesion are doing tasks that are not related to their responsibilities. +- Reasons for change: adding new features; correcting faults; refactoring. +- Isolate change by looking closely at the things that make the whole and separate them logically. +- Keep track of the dependencies by checking i.e. if a constructor/method has too many input parameters thus many dependencies. +- Refactor early (to design patterns) + +## Open Close Principle + +- Objects or entities should be open for extension, but closed for modification. +- Have a way to add new features but keep code in good standing. +- Existing code is immutable; extend with composition (avoid inheritance everywhere) +- Keep in mind future changes (scope meetings) + +## Liskov substitution principle + +- Let q(x) be a property provable about objects of x of type T. Then q(y) should be provable for objects y of type S where S is a subtype of T. (All this is stating is that every subclass/derived class should be substitutable for their base/parent class.) +- If not: class hierarchies get messy; Unit tests for the superclass never succeed for sub-class +- Keep base abstractions as simple and minimal as possible. +- Code reviews very helpful +- [Composition over inheritance](https://www.thoughtworks.com/insights/blog/composition-vs-inheritance-how-choose) + +## Interface segregation principle + +- A client should never be forced to implement an interface that it doesn't use or clients shouldn't be forced to depend on methods they do not use. +- Related to Single Responsibility Principle +- Keep interfaces thin or fine-grained/no unused methods => gives flexibility +- Thin interfaces are called *role interfaces* => easier to refactor +- If nescessary combine interfaces + +## Dependency inversion principle + +- Entities must depend on abstractions not on concretions. It states that the high level module must not depend on the low level module, but they should depend on abstractions. +http://wiki.c2.com/?DependencyInversionPrinciple + +- **loosely coupling** => no client has to be changed when code it depends has changed; better it has no knowlegde of the change. +- **testability and replaceability** components can be replaced with alternative with same service. +- Drawback: dependency inversion container is required and needs to be configured. Takes care of injection. \ No newline at end of file diff --git a/Vertaling/Engels.md b/Vertaling/Engels.md new file mode 100644 index 0000000..84bdc05 --- /dev/null +++ b/Vertaling/Engels.md @@ -0,0 +1,14 @@ +--- +title: Engels +updated: 2022-05-24 19:40:49Z +created: 2022-05-21 12:14:13Z +--- + +|Engels|Nederlands| +|-|-| +|pursuing|achtervolgen| +|unification|eenwording| +|provenance|herkomst| +|versatile|veelzijdig,veranderlijk, onstandvastig| + + diff --git a/Werk/Definities.md b/Werk/Definities.md new file mode 100644 index 0000000..d99bf99 --- /dev/null +++ b/Werk/Definities.md @@ -0,0 +1,29 @@ +--- +title: Definities +updated: 2022-05-19 19:40:12Z +created: 2022-05-19 19:10:30Z +--- + +## Toetsingscommissie Inzet Bevoegdheden (TIB) +Twee rechters of oud-rechters en een technisch expert. + + +## Voorwaarden voor bijzondere bevoegdheden +**Proportionaliteit**: staat belang in verhouding tot de inbreuk? +**Subsidiariteit**: is dit de beste manier om doel te bereiken? Of zijn er nog andere manieren? +**Doeltreffendheid**: bereik ik met inbreuk wat ik wil bereiken? +**Gerichtheid**: gericht zijn op het doel dat wordt nagestreefd in onderzoek +**Noodzakelijkheid**: Het moet noodzakelijk zijn om de bevoegdheid in te zetten, dus er moet een dreiging voor de nationale veiligheid bestaan. + +## Welke bijzondere bevoegdheden zijn er? +- het volgen en observeren van mensen en daarbij observatie- en registratiemiddelen inzetten; +- het inzetten van mensen om gegevens te verzamelen en bepaalde handelingen te verrichten; +- het oprichten en inzetten van rechtspersonen (bedrijven, organisaties) om operaties te ondersteunen; +- het doorzoeken van woningen en andere besloten plaatsen en voorwerpen; +- het openen van brieven, e-mails en andere geadresseerde zendingen; +- het [opvragen van gegevens](https://www.aivd.nl/onderwerpen/gegevens-van-communicatiediensten) bij communicatiediensten; +- het [onderzoeken van de inhoud van computers](https://www.aivd.nl/onderwerpen/hacken) (en andere zogenaamde geautomatiseerde werken); +- het tappen, ontvangen, opnemen en [afluisteren van gesprekken](https://www.aivd.nl/onderwerpen/afluisteren), telecommunicatie (zoals telefoon) of andere vormen van communicatie; +- het [onderzoeken van datastromen](https://www.aivd.nl/onderwerpen/onderzoeksopdrachtgerichte-interceptie-oog) uit de ether of via internetkabels; +- het gebruiken van [DNA-profielen](https://www.aivd.nl/onderwerpen/dna-onderzoek) om personen te identificeren. + diff --git a/Werk/Onderzoeksopdrachtgerichte interceptie (OOG).md b/Werk/Onderzoeksopdrachtgerichte interceptie (OOG).md new file mode 100644 index 0000000..c07ac11 --- /dev/null +++ b/Werk/Onderzoeksopdrachtgerichte interceptie (OOG).md @@ -0,0 +1,26 @@ +--- +title: Onderzoeksopdrachtgerichte interceptie (OOG) +updated: 2022-05-19 19:34:37Z +created: 2022-05-19 19:19:38Z +--- + +### OOG-interceptie alleen inzetten als dit nodig is voor het onderzoeken van dreigingen voor de nationale veiligheid => [Geïntegreerde Aanwijzing (GA)](https://www.aivd.nl/onderwerpen/sturing-op-het-werk-van-de-aivd) + +## OOG-interceptie, drie fasen met ieder hun eigen waarborgen en TIB toestemmingen: +- Filtering + - welke satelliet of welke acceslocatie (toestemming van de minister nodig en toetsing TIB) + - bepaalde 'link' bij een satelliet, een frequentie van radioverkeer of een specifieke fiber in een datakabel (toestemming van de minister nodig en toetsing TIB) + - negatieve en positieve filters +- Selectie + - scheiden in metadata en inhoud + - Op data die inhoud bevatten passen we selectie toe voordat we dit kunnen gebruiken in ons onderzoek (selectiecriteria bv telelfoonnummer) (toestemming van de minister nodig en toetsing TIB) +- Analyse + - metadata analyseren we op basis van bepaalde kenmerken (toestemming van de minister nodig en toetsing TIB) + - bewaartermijnen: + - kabeldata: 1 jaar tijd voor beoordeling op relevantie. + - bij nog niet relevant verlenging, via minister, met 1 jaar. Mag 2x. + - etherdata: 3 jaar tijd voor beoordeling op relevantie. + +Data die relevant is beoordeeld mag zolang worden bewaard als deze betekenis heeft. + +[Source](https://www.aivd.nl/onderwerpen/onderzoeksopdrachtgerichte-interceptie-oog) \ No newline at end of file diff --git a/_resources/0532ef04db4483bf2be14245b2c02b4b.png b/_resources/0532ef04db4483bf2be14245b2c02b4b.png new file mode 100644 index 0000000..bfafe38 Binary files /dev/null and b/_resources/0532ef04db4483bf2be14245b2c02b4b.png differ diff --git a/_resources/0745b249204a6a1ab9b5aaafaa18d53b.png b/_resources/0745b249204a6a1ab9b5aaafaa18d53b.png new file mode 100644 index 0000000..7a8363c Binary files /dev/null and b/_resources/0745b249204a6a1ab9b5aaafaa18d53b.png differ diff --git a/_resources/0b8629bbcc1c0bf173fcefc7ea92df53.png b/_resources/0b8629bbcc1c0bf173fcefc7ea92df53.png new file mode 100644 index 0000000..92d0025 Binary files /dev/null and b/_resources/0b8629bbcc1c0bf173fcefc7ea92df53.png differ diff --git a/_resources/0c0746feb4d1a4cb7802f32da7da5a43.png b/_resources/0c0746feb4d1a4cb7802f32da7da5a43.png new file mode 100644 index 0000000..8db5e09 Binary files /dev/null and b/_resources/0c0746feb4d1a4cb7802f32da7da5a43.png differ diff --git a/_resources/0ceb8df673446ebaaa7f858751a50669.png b/_resources/0ceb8df673446ebaaa7f858751a50669.png new file mode 100644 index 0000000..da4e98e Binary files /dev/null and b/_resources/0ceb8df673446ebaaa7f858751a50669.png differ diff --git a/_resources/0dbbbe727224fc349edc14517e3b93c9.png b/_resources/0dbbbe727224fc349edc14517e3b93c9.png new file mode 100644 index 0000000..cced5e8 Binary files /dev/null and b/_resources/0dbbbe727224fc349edc14517e3b93c9.png differ diff --git a/_resources/0dc381da545d4b7f9885e08555d4445a.png b/_resources/0dc381da545d4b7f9885e08555d4445a.png new file mode 100644 index 0000000..888d6bc Binary files /dev/null and b/_resources/0dc381da545d4b7f9885e08555d4445a.png differ diff --git a/_resources/0e5c55b7a519831b5ba0393544641782.png b/_resources/0e5c55b7a519831b5ba0393544641782.png new file mode 100644 index 0000000..d2619e9 Binary files /dev/null and b/_resources/0e5c55b7a519831b5ba0393544641782.png differ diff --git a/_resources/1166787a12cf4602040c27e3863abb43.png b/_resources/1166787a12cf4602040c27e3863abb43.png new file mode 100644 index 0000000..dc5b2a4 Binary files /dev/null and b/_resources/1166787a12cf4602040c27e3863abb43.png differ diff --git a/_resources/1238eb4c4d458a3679c68dff091d8e0c.png b/_resources/1238eb4c4d458a3679c68dff091d8e0c.png new file mode 100644 index 0000000..672dba0 Binary files /dev/null and b/_resources/1238eb4c4d458a3679c68dff091d8e0c.png differ diff --git a/_resources/1798e9a44636860cd20db3b66ede4949.png b/_resources/1798e9a44636860cd20db3b66ede4949.png new file mode 100644 index 0000000..fe1808a Binary files /dev/null and b/_resources/1798e9a44636860cd20db3b66ede4949.png differ diff --git a/_resources/1863886204ff15acbf0544007ff3c91a.png b/_resources/1863886204ff15acbf0544007ff3c91a.png new file mode 100644 index 0000000..4871fd1 Binary files /dev/null and b/_resources/1863886204ff15acbf0544007ff3c91a.png differ diff --git a/_resources/193bfaaed851ce4eef6d43f0a20c03a7.png b/_resources/193bfaaed851ce4eef6d43f0a20c03a7.png new file mode 100644 index 0000000..b2447c0 Binary files /dev/null and b/_resources/193bfaaed851ce4eef6d43f0a20c03a7.png differ diff --git a/_resources/19949739424c8962762275254b28a831.png b/_resources/19949739424c8962762275254b28a831.png new file mode 100644 index 0000000..a1f759a Binary files /dev/null and b/_resources/19949739424c8962762275254b28a831.png differ diff --git a/_resources/1ce08014a43470c07e5314f1d69c6771.png b/_resources/1ce08014a43470c07e5314f1d69c6771.png new file mode 100644 index 0000000..eade2fa Binary files /dev/null and b/_resources/1ce08014a43470c07e5314f1d69c6771.png differ diff --git a/_resources/200ebd8d5ccb744efcc58af1fa67988d.png b/_resources/200ebd8d5ccb744efcc58af1fa67988d.png new file mode 100644 index 0000000..b2d5594 Binary files /dev/null and b/_resources/200ebd8d5ccb744efcc58af1fa67988d.png differ diff --git a/_resources/25f981495ea54faccfe5ea58cd1f8b75.png b/_resources/25f981495ea54faccfe5ea58cd1f8b75.png new file mode 100644 index 0000000..b04c7f9 Binary files /dev/null and b/_resources/25f981495ea54faccfe5ea58cd1f8b75.png differ diff --git a/_resources/26d5943bae46c8a9bc9a4210eb56ed45.png b/_resources/26d5943bae46c8a9bc9a4210eb56ed45.png new file mode 100644 index 0000000..d7b29b4 Binary files /dev/null and b/_resources/26d5943bae46c8a9bc9a4210eb56ed45.png differ diff --git a/_resources/28f3c610f7afc0d7715e6bfa470edcdd.png b/_resources/28f3c610f7afc0d7715e6bfa470edcdd.png new file mode 100644 index 0000000..9e0e357 Binary files /dev/null and b/_resources/28f3c610f7afc0d7715e6bfa470edcdd.png differ diff --git a/_resources/2e79a268b8d9ef0e61db6ea008bafd7d.png b/_resources/2e79a268b8d9ef0e61db6ea008bafd7d.png new file mode 100644 index 0000000..593758a Binary files /dev/null and b/_resources/2e79a268b8d9ef0e61db6ea008bafd7d.png differ diff --git a/_resources/2fa4ce2967893077a569d372aed8a3ff.png b/_resources/2fa4ce2967893077a569d372aed8a3ff.png new file mode 100644 index 0000000..16add78 Binary files /dev/null and b/_resources/2fa4ce2967893077a569d372aed8a3ff.png differ diff --git a/_resources/31156c4015ea6f9c5ae11c6a02d28cbe.png b/_resources/31156c4015ea6f9c5ae11c6a02d28cbe.png new file mode 100644 index 0000000..ff0701c Binary files /dev/null and b/_resources/31156c4015ea6f9c5ae11c6a02d28cbe.png differ diff --git a/_resources/317aa12b279ad4a9b0e289d1c654c4a3.png b/_resources/317aa12b279ad4a9b0e289d1c654c4a3.png new file mode 100644 index 0000000..36ea6f6 Binary files /dev/null and b/_resources/317aa12b279ad4a9b0e289d1c654c4a3.png differ diff --git a/_resources/3393db98a463262eb4c2453a6140b984.png b/_resources/3393db98a463262eb4c2453a6140b984.png new file mode 100644 index 0000000..528a2ee Binary files /dev/null and b/_resources/3393db98a463262eb4c2453a6140b984.png differ diff --git a/_resources/34262aff59c5f5dd9a413b2b3d74629a.png b/_resources/34262aff59c5f5dd9a413b2b3d74629a.png new file mode 100644 index 0000000..76c7107 Binary files /dev/null and b/_resources/34262aff59c5f5dd9a413b2b3d74629a.png differ diff --git a/_resources/35424d82fe826bba9b7f251153ef6278.png b/_resources/35424d82fe826bba9b7f251153ef6278.png new file mode 100644 index 0000000..0e0e99b Binary files /dev/null and b/_resources/35424d82fe826bba9b7f251153ef6278.png differ diff --git a/_resources/35ae293edf8fad30686a64bbe5dcc90a.png b/_resources/35ae293edf8fad30686a64bbe5dcc90a.png new file mode 100644 index 0000000..53e1616 Binary files /dev/null and b/_resources/35ae293edf8fad30686a64bbe5dcc90a.png differ diff --git a/_resources/362653ef1df287f348947de381643f48.png b/_resources/362653ef1df287f348947de381643f48.png new file mode 100644 index 0000000..56a06a1 Binary files /dev/null and b/_resources/362653ef1df287f348947de381643f48.png differ diff --git a/_resources/36ea2fef8a287ca026801e2d45632c20.png b/_resources/36ea2fef8a287ca026801e2d45632c20.png new file mode 100644 index 0000000..43b8c7b Binary files /dev/null and b/_resources/36ea2fef8a287ca026801e2d45632c20.png differ diff --git a/_resources/396364a8c39247681880d65a1f9b9a8b.png b/_resources/396364a8c39247681880d65a1f9b9a8b.png new file mode 100644 index 0000000..cdbbdac Binary files /dev/null and b/_resources/396364a8c39247681880d65a1f9b9a8b.png differ diff --git a/_resources/3a3ffb85892084804f4ad8c23681d1b2.png b/_resources/3a3ffb85892084804f4ad8c23681d1b2.png new file mode 100644 index 0000000..a3ce568 Binary files /dev/null and b/_resources/3a3ffb85892084804f4ad8c23681d1b2.png differ diff --git a/_resources/3f0188b20328cc489d50b321df8d6bc7.png b/_resources/3f0188b20328cc489d50b321df8d6bc7.png new file mode 100644 index 0000000..173a314 Binary files /dev/null and b/_resources/3f0188b20328cc489d50b321df8d6bc7.png differ diff --git a/_resources/4097a690bcdbf2af251898baec7a4adf.png b/_resources/4097a690bcdbf2af251898baec7a4adf.png new file mode 100644 index 0000000..532ab87 Binary files /dev/null and b/_resources/4097a690bcdbf2af251898baec7a4adf.png differ diff --git a/_resources/410d3a1816f7f58c8d812d7047ada137.png b/_resources/410d3a1816f7f58c8d812d7047ada137.png new file mode 100644 index 0000000..1b75374 Binary files /dev/null and b/_resources/410d3a1816f7f58c8d812d7047ada137.png differ diff --git a/_resources/418daa8979836e67e544c4b275820715.png b/_resources/418daa8979836e67e544c4b275820715.png new file mode 100644 index 0000000..5f9b31a Binary files /dev/null and b/_resources/418daa8979836e67e544c4b275820715.png differ diff --git a/_resources/43d7aa7e1b2fb5703bc8874d6295cf1c.png b/_resources/43d7aa7e1b2fb5703bc8874d6295cf1c.png new file mode 100644 index 0000000..938e6e0 Binary files /dev/null and b/_resources/43d7aa7e1b2fb5703bc8874d6295cf1c.png differ diff --git a/_resources/46f3e8c2b3038d9c4b19536c9cdd9a21.png b/_resources/46f3e8c2b3038d9c4b19536c9cdd9a21.png new file mode 100644 index 0000000..9c0defb Binary files /dev/null and b/_resources/46f3e8c2b3038d9c4b19536c9cdd9a21.png differ diff --git a/_resources/47073357d36804a22ad703c90cb541ce.png b/_resources/47073357d36804a22ad703c90cb541ce.png new file mode 100644 index 0000000..9325b9a Binary files /dev/null and b/_resources/47073357d36804a22ad703c90cb541ce.png differ diff --git a/_resources/48751b057b60e03ec51f64e3235fa1b3.png b/_resources/48751b057b60e03ec51f64e3235fa1b3.png new file mode 100644 index 0000000..24ad590 Binary files /dev/null and b/_resources/48751b057b60e03ec51f64e3235fa1b3.png differ diff --git a/_resources/4bd6a4b0fe00fd4735c0efd1b01892a1.png b/_resources/4bd6a4b0fe00fd4735c0efd1b01892a1.png new file mode 100644 index 0000000..f6b5640 Binary files /dev/null and b/_resources/4bd6a4b0fe00fd4735c0efd1b01892a1.png differ diff --git a/_resources/4c3b142e5e73a6ac715600d0331150c2.png b/_resources/4c3b142e5e73a6ac715600d0331150c2.png new file mode 100644 index 0000000..bab0b89 Binary files /dev/null and b/_resources/4c3b142e5e73a6ac715600d0331150c2.png differ diff --git a/_resources/4c5a7cb8f10e6e2cc67ccfb4681679b7.png b/_resources/4c5a7cb8f10e6e2cc67ccfb4681679b7.png new file mode 100644 index 0000000..2383b6d Binary files /dev/null and b/_resources/4c5a7cb8f10e6e2cc67ccfb4681679b7.png differ diff --git a/_resources/4d44a731b2dcebdb8e5210d5f6378b45.png b/_resources/4d44a731b2dcebdb8e5210d5f6378b45.png new file mode 100644 index 0000000..857b62d Binary files /dev/null and b/_resources/4d44a731b2dcebdb8e5210d5f6378b45.png differ diff --git a/_resources/4e05321688bdc9bd53da7809ff1e4961.png b/_resources/4e05321688bdc9bd53da7809ff1e4961.png new file mode 100644 index 0000000..d471bfc Binary files /dev/null and b/_resources/4e05321688bdc9bd53da7809ff1e4961.png differ diff --git a/_resources/5151c1c2b55de604999b2b6f38822b08.png b/_resources/5151c1c2b55de604999b2b6f38822b08.png new file mode 100644 index 0000000..d4c1b20 Binary files /dev/null and b/_resources/5151c1c2b55de604999b2b6f38822b08.png differ diff --git a/_resources/51c2fc2521e7435aba198a5c23584999.png b/_resources/51c2fc2521e7435aba198a5c23584999.png new file mode 100644 index 0000000..6089a5d Binary files /dev/null and b/_resources/51c2fc2521e7435aba198a5c23584999.png differ diff --git a/_resources/5362730add08cb32ca4de62973425703.png b/_resources/5362730add08cb32ca4de62973425703.png new file mode 100644 index 0000000..3828643 Binary files /dev/null and b/_resources/5362730add08cb32ca4de62973425703.png differ diff --git a/_resources/536ce7d9e4c324339ba03a53d7eaaec9.png b/_resources/536ce7d9e4c324339ba03a53d7eaaec9.png new file mode 100644 index 0000000..801566b Binary files /dev/null and b/_resources/536ce7d9e4c324339ba03a53d7eaaec9.png differ diff --git a/_resources/5563a5e13ec8cdd9e44831e7209880a2.png b/_resources/5563a5e13ec8cdd9e44831e7209880a2.png new file mode 100644 index 0000000..e935a2f Binary files /dev/null and b/_resources/5563a5e13ec8cdd9e44831e7209880a2.png differ diff --git a/_resources/56dff0bf25437b4d77b8a2504ff17ae8.png b/_resources/56dff0bf25437b4d77b8a2504ff17ae8.png new file mode 100644 index 0000000..fd879fd Binary files /dev/null and b/_resources/56dff0bf25437b4d77b8a2504ff17ae8.png differ diff --git a/_resources/57f223b1256deb84e40fbefc1bc5960a.png b/_resources/57f223b1256deb84e40fbefc1bc5960a.png new file mode 100644 index 0000000..619bdf9 Binary files /dev/null and b/_resources/57f223b1256deb84e40fbefc1bc5960a.png differ diff --git a/_resources/5815ff14a02f76a5118a6ef8f86ed048.png b/_resources/5815ff14a02f76a5118a6ef8f86ed048.png new file mode 100644 index 0000000..e5ca8c7 Binary files /dev/null and b/_resources/5815ff14a02f76a5118a6ef8f86ed048.png differ diff --git a/_resources/5908a4d876bce77f6cc35e42ca230be3.png b/_resources/5908a4d876bce77f6cc35e42ca230be3.png new file mode 100644 index 0000000..de93465 Binary files /dev/null and b/_resources/5908a4d876bce77f6cc35e42ca230be3.png differ diff --git a/_resources/5_physical_plan.png b/_resources/5_physical_plan.png new file mode 100644 index 0000000..c438e6d Binary files /dev/null and b/_resources/5_physical_plan.png differ diff --git a/_resources/5a61dcd70f5bb58e422ce7b3ef30f1b2.png b/_resources/5a61dcd70f5bb58e422ce7b3ef30f1b2.png new file mode 100644 index 0000000..67bab0d Binary files /dev/null and b/_resources/5a61dcd70f5bb58e422ce7b3ef30f1b2.png differ diff --git a/_resources/5b5bd4d599ad5e479af873c9f714ee7b.png b/_resources/5b5bd4d599ad5e479af873c9f714ee7b.png new file mode 100644 index 0000000..61cc330 Binary files /dev/null and b/_resources/5b5bd4d599ad5e479af873c9f714ee7b.png differ diff --git a/_resources/5ea764eaeeb247f7b43501c4d6e11653.png b/_resources/5ea764eaeeb247f7b43501c4d6e11653.png new file mode 100644 index 0000000..6e950db Binary files /dev/null and b/_resources/5ea764eaeeb247f7b43501c4d6e11653.png differ diff --git a/_resources/5f01121f49a2b85234a4dde5f3c243fa.png b/_resources/5f01121f49a2b85234a4dde5f3c243fa.png new file mode 100644 index 0000000..2646e12 Binary files /dev/null and b/_resources/5f01121f49a2b85234a4dde5f3c243fa.png differ diff --git a/_resources/5f9685eb06bc6b913e29821e3c135f78.png b/_resources/5f9685eb06bc6b913e29821e3c135f78.png new file mode 100644 index 0000000..4a0024e Binary files /dev/null and b/_resources/5f9685eb06bc6b913e29821e3c135f78.png differ diff --git a/_resources/60b4e3ed0ebd1c3fb65b18d812f5f8fb.png b/_resources/60b4e3ed0ebd1c3fb65b18d812f5f8fb.png new file mode 100644 index 0000000..a51282f Binary files /dev/null and b/_resources/60b4e3ed0ebd1c3fb65b18d812f5f8fb.png differ diff --git a/_resources/626a552aa3d1eb5331b7e1a0ed5161af.png b/_resources/626a552aa3d1eb5331b7e1a0ed5161af.png new file mode 100644 index 0000000..d0221b3 Binary files /dev/null and b/_resources/626a552aa3d1eb5331b7e1a0ed5161af.png differ diff --git a/_resources/65d46691cee632fc49a1054a983fba7c.png b/_resources/65d46691cee632fc49a1054a983fba7c.png new file mode 100644 index 0000000..aa0761c Binary files /dev/null and b/_resources/65d46691cee632fc49a1054a983fba7c.png differ diff --git a/_resources/67abe953280b3a8006e5a19d4e094040.png b/_resources/67abe953280b3a8006e5a19d4e094040.png new file mode 100644 index 0000000..bed5ab3 Binary files /dev/null and b/_resources/67abe953280b3a8006e5a19d4e094040.png differ diff --git a/_resources/69b4c46435ed52c1fe5be0ba6a074be5.png b/_resources/69b4c46435ed52c1fe5be0ba6a074be5.png new file mode 100644 index 0000000..bc4e816 Binary files /dev/null and b/_resources/69b4c46435ed52c1fe5be0ba6a074be5.png differ diff --git a/_resources/6af1399567c87fcda04a6414efbe18bf.png b/_resources/6af1399567c87fcda04a6414efbe18bf.png new file mode 100644 index 0000000..10dc951 Binary files /dev/null and b/_resources/6af1399567c87fcda04a6414efbe18bf.png differ diff --git a/_resources/703c231dfc0b7336315963917d74b038.png b/_resources/703c231dfc0b7336315963917d74b038.png new file mode 100644 index 0000000..57aa68a Binary files /dev/null and b/_resources/703c231dfc0b7336315963917d74b038.png differ diff --git a/_resources/73822f556381b2d599822c18a4e09d83.png b/_resources/73822f556381b2d599822c18a4e09d83.png new file mode 100644 index 0000000..5c5f138 Binary files /dev/null and b/_resources/73822f556381b2d599822c18a4e09d83.png differ diff --git a/_resources/765bc3651f87f986301649e1efafd128.png b/_resources/765bc3651f87f986301649e1efafd128.png new file mode 100644 index 0000000..cfdde84 Binary files /dev/null and b/_resources/765bc3651f87f986301649e1efafd128.png differ diff --git a/_resources/77e6428cf3d2bff19c7215e2915ca3c4.png b/_resources/77e6428cf3d2bff19c7215e2915ca3c4.png new file mode 100644 index 0000000..e082dd2 Binary files /dev/null and b/_resources/77e6428cf3d2bff19c7215e2915ca3c4.png differ diff --git a/_resources/7ad6809dc9b770b2bb4967f5c8795b2f.png b/_resources/7ad6809dc9b770b2bb4967f5c8795b2f.png new file mode 100644 index 0000000..0d6a063 Binary files /dev/null and b/_resources/7ad6809dc9b770b2bb4967f5c8795b2f.png differ diff --git a/_resources/7b7de3102acdf026a8f7d05aba8e7217.png b/_resources/7b7de3102acdf026a8f7d05aba8e7217.png new file mode 100644 index 0000000..7e9c100 Binary files /dev/null and b/_resources/7b7de3102acdf026a8f7d05aba8e7217.png differ diff --git a/_resources/7e3dd277a7935550cabb842d0b3e91ef.png b/_resources/7e3dd277a7935550cabb842d0b3e91ef.png new file mode 100644 index 0000000..6b54ad2 Binary files /dev/null and b/_resources/7e3dd277a7935550cabb842d0b3e91ef.png differ diff --git a/_resources/846c953521751f708bd680556dc9ae0b.png b/_resources/846c953521751f708bd680556dc9ae0b.png new file mode 100644 index 0000000..ceb893c Binary files /dev/null and b/_resources/846c953521751f708bd680556dc9ae0b.png differ diff --git a/_resources/87bb8cb9d5ec12045bbbf38c08b8c7a0.png b/_resources/87bb8cb9d5ec12045bbbf38c08b8c7a0.png new file mode 100644 index 0000000..f16a8a1 Binary files /dev/null and b/_resources/87bb8cb9d5ec12045bbbf38c08b8c7a0.png differ diff --git a/_resources/87df2df0f990da8682065eeb7f8381a9.png b/_resources/87df2df0f990da8682065eeb7f8381a9.png new file mode 100644 index 0000000..2ea670e Binary files /dev/null and b/_resources/87df2df0f990da8682065eeb7f8381a9.png differ diff --git a/_resources/89ef06b3ace7055254b5a4d1b2db1663.png b/_resources/89ef06b3ace7055254b5a4d1b2db1663.png new file mode 100644 index 0000000..70d9322 Binary files /dev/null and b/_resources/89ef06b3ace7055254b5a4d1b2db1663.png differ diff --git a/_resources/8bc3cae2d26662dcf3d103e60e365a3f.png b/_resources/8bc3cae2d26662dcf3d103e60e365a3f.png new file mode 100644 index 0000000..db3b1e0 Binary files /dev/null and b/_resources/8bc3cae2d26662dcf3d103e60e365a3f.png differ diff --git a/_resources/8bc6b50bc979e6571d513235dd90457d.png b/_resources/8bc6b50bc979e6571d513235dd90457d.png new file mode 100644 index 0000000..df2f71f Binary files /dev/null and b/_resources/8bc6b50bc979e6571d513235dd90457d.png differ diff --git a/_resources/8cc8d717ee428ffeb48a396d804b91de.png b/_resources/8cc8d717ee428ffeb48a396d804b91de.png new file mode 100644 index 0000000..aab3deb Binary files /dev/null and b/_resources/8cc8d717ee428ffeb48a396d804b91de.png differ diff --git a/_resources/8cf3db7009d5518b37d9b0deb9b88dea.png b/_resources/8cf3db7009d5518b37d9b0deb9b88dea.png new file mode 100644 index 0000000..8bdd472 Binary files /dev/null and b/_resources/8cf3db7009d5518b37d9b0deb9b88dea.png differ diff --git a/_resources/90294faf40d4ca7b98565abeec1db421.png b/_resources/90294faf40d4ca7b98565abeec1db421.png new file mode 100644 index 0000000..a7a639a Binary files /dev/null and b/_resources/90294faf40d4ca7b98565abeec1db421.png differ diff --git a/_resources/9106b716ae276078eea921e3e03078ee.png b/_resources/9106b716ae276078eea921e3e03078ee.png new file mode 100644 index 0000000..8511af0 Binary files /dev/null and b/_resources/9106b716ae276078eea921e3e03078ee.png differ diff --git a/_resources/91c43645ccabeee0047757424141fb5d.png b/_resources/91c43645ccabeee0047757424141fb5d.png new file mode 100644 index 0000000..774ea2b Binary files /dev/null and b/_resources/91c43645ccabeee0047757424141fb5d.png differ diff --git a/_resources/920bc1ebbbe823a40736ed122e695036.png b/_resources/920bc1ebbbe823a40736ed122e695036.png new file mode 100644 index 0000000..67a7823 Binary files /dev/null and b/_resources/920bc1ebbbe823a40736ed122e695036.png differ diff --git a/_resources/92c4ab63a9cc743176d498e462af41af.png b/_resources/92c4ab63a9cc743176d498e462af41af.png new file mode 100644 index 0000000..258a4a2 Binary files /dev/null and b/_resources/92c4ab63a9cc743176d498e462af41af.png differ diff --git a/_resources/94a60b4329d9d14e300dd2f806af1521.png b/_resources/94a60b4329d9d14e300dd2f806af1521.png new file mode 100644 index 0000000..08917b8 Binary files /dev/null and b/_resources/94a60b4329d9d14e300dd2f806af1521.png differ diff --git a/_resources/95570cc80acedfbfa19a48f3f89b9024.png b/_resources/95570cc80acedfbfa19a48f3f89b9024.png new file mode 100644 index 0000000..c055ed7 Binary files /dev/null and b/_resources/95570cc80acedfbfa19a48f3f89b9024.png differ diff --git a/_resources/95d94272c1cec38e84f0dd6d9a5c9e48.png b/_resources/95d94272c1cec38e84f0dd6d9a5c9e48.png new file mode 100644 index 0000000..9e0d126 Binary files /dev/null and b/_resources/95d94272c1cec38e84f0dd6d9a5c9e48.png differ diff --git a/_resources/9816005cb285fba7d59895ab9686676c.png b/_resources/9816005cb285fba7d59895ab9686676c.png new file mode 100644 index 0000000..8e37dd2 Binary files /dev/null and b/_resources/9816005cb285fba7d59895ab9686676c.png differ diff --git a/_resources/981ea34418a595b422aab0b0df23f4b6.png b/_resources/981ea34418a595b422aab0b0df23f4b6.png new file mode 100644 index 0000000..6f17bb9 Binary files /dev/null and b/_resources/981ea34418a595b422aab0b0df23f4b6.png differ diff --git a/_resources/9948507e5860dd40b34a9f10e6b370c2.png b/_resources/9948507e5860dd40b34a9f10e6b370c2.png new file mode 100644 index 0000000..7153b9a Binary files /dev/null and b/_resources/9948507e5860dd40b34a9f10e6b370c2.png differ diff --git a/_resources/99bd030fd518baa2ca26a85b313e23f0.png b/_resources/99bd030fd518baa2ca26a85b313e23f0.png new file mode 100644 index 0000000..21102b6 Binary files /dev/null and b/_resources/99bd030fd518baa2ca26a85b313e23f0.png differ diff --git a/_resources/9c2b3ab61d6331389b844684b029f52f.png b/_resources/9c2b3ab61d6331389b844684b029f52f.png new file mode 100644 index 0000000..3ef7a7b Binary files /dev/null and b/_resources/9c2b3ab61d6331389b844684b029f52f.png differ diff --git a/_resources/BRNN-ner.png b/_resources/BRNN-ner.png new file mode 100644 index 0000000..e3c3696 Binary files /dev/null and b/_resources/BRNN-ner.png differ diff --git a/_resources/BRNN.png b/_resources/BRNN.png new file mode 100644 index 0000000..c77ca96 Binary files /dev/null and b/_resources/BRNN.png differ diff --git a/_resources/CTC-rnn.png b/_resources/CTC-rnn.png new file mode 100644 index 0000000..7a8004b Binary files /dev/null and b/_resources/CTC-rnn.png differ diff --git a/_resources/CatalystOptimizer-1.png b/_resources/CatalystOptimizer-1.png new file mode 100644 index 0000000..ff4c0a7 Binary files /dev/null and b/_resources/CatalystOptimizer-1.png differ diff --git a/_resources/CatalystOptimizer.png b/_resources/CatalystOptimizer.png new file mode 100644 index 0000000..ff4c0a7 Binary files /dev/null and b/_resources/CatalystOptimizer.png differ diff --git a/_resources/CoreHadoopEcosystem.png b/_resources/CoreHadoopEcosystem.png new file mode 100644 index 0000000..bd3a7a8 Binary files /dev/null and b/_resources/CoreHadoopEcosystem.png differ diff --git a/_resources/DRNN.png b/_resources/DRNN.png new file mode 100644 index 0000000..f00fc9c Binary files /dev/null and b/_resources/DRNN.png differ diff --git a/_resources/FlumeExample.png b/_resources/FlumeExample.png new file mode 100644 index 0000000..e5a4629 Binary files /dev/null and b/_resources/FlumeExample.png differ diff --git a/_resources/GRU.png b/_resources/GRU.png new file mode 100644 index 0000000..eb342b6 Binary files /dev/null and b/_resources/GRU.png differ diff --git a/_resources/HDFS_Read_Mechanism.png b/_resources/HDFS_Read_Mechanism.png new file mode 100644 index 0000000..5c8223b Binary files /dev/null and b/_resources/HDFS_Read_Mechanism.png differ diff --git a/_resources/HDFS_Write_mechanism.jpeg b/_resources/HDFS_Write_mechanism.jpeg new file mode 100644 index 0000000..013594d Binary files /dev/null and b/_resources/HDFS_Write_mechanism.jpeg differ diff --git a/_resources/HadoopLogicalView.jpg b/_resources/HadoopLogicalView.jpg new file mode 100644 index 0000000..6c41034 Binary files /dev/null and b/_resources/HadoopLogicalView.jpg differ diff --git a/_resources/HadoopMapperReduce_conceptionally.png b/_resources/HadoopMapperReduce_conceptionally.png new file mode 100644 index 0000000..d9347d5 Binary files /dev/null and b/_resources/HadoopMapperReduce_conceptionally.png differ diff --git a/_resources/IaaS.png b/_resources/IaaS.png new file mode 100644 index 0000000..cd86a37 Binary files /dev/null and b/_resources/IaaS.png differ diff --git a/_resources/LSTM-units.png b/_resources/LSTM-units.png new file mode 100644 index 0000000..a7568c5 Binary files /dev/null and b/_resources/LSTM-units.png differ diff --git a/_resources/LSTM.png b/_resources/LSTM.png new file mode 100644 index 0000000..1cca596 Binary files /dev/null and b/_resources/LSTM.png differ diff --git a/_resources/LogicalPlanSpark-1.png b/_resources/LogicalPlanSpark-1.png new file mode 100644 index 0000000..d2c15b6 Binary files /dev/null and b/_resources/LogicalPlanSpark-1.png differ diff --git a/_resources/LogicalPlanSpark.png b/_resources/LogicalPlanSpark.png new file mode 100644 index 0000000..d2c15b6 Binary files /dev/null and b/_resources/LogicalPlanSpark.png differ diff --git a/_resources/OverviewMapReduceMasterNodes.png b/_resources/OverviewMapReduceMasterNodes.png new file mode 100644 index 0000000..4a735ed Binary files /dev/null and b/_resources/OverviewMapReduceMasterNodes.png differ diff --git a/_resources/PaaS.png b/_resources/PaaS.png new file mode 100644 index 0000000..5111743 Binary files /dev/null and b/_resources/PaaS.png differ diff --git a/_resources/PhysicalPlanSpark-1.png b/_resources/PhysicalPlanSpark-1.png new file mode 100644 index 0000000..c98c811 Binary files /dev/null and b/_resources/PhysicalPlanSpark-1.png differ diff --git a/_resources/PhysicalPlanSpark.png b/_resources/PhysicalPlanSpark.png new file mode 100644 index 0000000..c98c811 Binary files /dev/null and b/_resources/PhysicalPlanSpark.png differ diff --git a/_resources/ReadSortTakeDataframe-1.png b/_resources/ReadSortTakeDataframe-1.png new file mode 100644 index 0000000..3aa29c7 Binary files /dev/null and b/_resources/ReadSortTakeDataframe-1.png differ diff --git a/_resources/ReadSortTakeDataframe.png b/_resources/ReadSortTakeDataframe.png new file mode 100644 index 0000000..3aa29c7 Binary files /dev/null and b/_resources/ReadSortTakeDataframe.png differ diff --git a/_resources/SaaS.png b/_resources/SaaS.png new file mode 100644 index 0000000..df6c84e Binary files /dev/null and b/_resources/SaaS.png differ diff --git a/_resources/a157a98c23f8955d06c56f8be279c074.png b/_resources/a157a98c23f8955d06c56f8be279c074.png new file mode 100644 index 0000000..35b0068 Binary files /dev/null and b/_resources/a157a98c23f8955d06c56f8be279c074.png differ diff --git a/_resources/a435b4325f8a97c83018933cc44dff54.png b/_resources/a435b4325f8a97c83018933cc44dff54.png new file mode 100644 index 0000000..9fa746b Binary files /dev/null and b/_resources/a435b4325f8a97c83018933cc44dff54.png differ diff --git a/_resources/a546793b2366edc935ce4a1d98db7f1e.png b/_resources/a546793b2366edc935ce4a1d98db7f1e.png new file mode 100644 index 0000000..6cb9663 Binary files /dev/null and b/_resources/a546793b2366edc935ce4a1d98db7f1e.png differ diff --git a/_resources/a6d9be269ce6bc83c9f5d824ef667013.png b/_resources/a6d9be269ce6bc83c9f5d824ef667013.png new file mode 100644 index 0000000..76f0537 Binary files /dev/null and b/_resources/a6d9be269ce6bc83c9f5d824ef667013.png differ diff --git a/_resources/a6ea2a07d58fac8a6739c7379c1b92f6.png b/_resources/a6ea2a07d58fac8a6739c7379c1b92f6.png new file mode 100644 index 0000000..f6053ce Binary files /dev/null and b/_resources/a6ea2a07d58fac8a6739c7379c1b92f6.png differ diff --git a/_resources/a6f83a73449bec9c309c2cab27c24f5e.png b/_resources/a6f83a73449bec9c309c2cab27c24f5e.png new file mode 100644 index 0000000..1e91e1b Binary files /dev/null and b/_resources/a6f83a73449bec9c309c2cab27c24f5e.png differ diff --git a/_resources/a8e058c2e06084485b3140b75a20e78e.png b/_resources/a8e058c2e06084485b3140b75a20e78e.png new file mode 100644 index 0000000..4c0a1f0 Binary files /dev/null and b/_resources/a8e058c2e06084485b3140b75a20e78e.png differ diff --git a/_resources/a99bbee6299b41776e45d4af26288c05.png b/_resources/a99bbee6299b41776e45d4af26288c05.png new file mode 100644 index 0000000..203ad04 Binary files /dev/null and b/_resources/a99bbee6299b41776e45d4af26288c05.png differ diff --git a/_resources/aca7155a4c7e4f2b982378cbb2d37400.png b/_resources/aca7155a4c7e4f2b982378cbb2d37400.png new file mode 100644 index 0000000..f56c9f1 Binary files /dev/null and b/_resources/aca7155a4c7e4f2b982378cbb2d37400.png differ diff --git a/_resources/ace369b638966681b9558c42e25dd0b4.png b/_resources/ace369b638966681b9558c42e25dd0b4.png new file mode 100644 index 0000000..ae2058b Binary files /dev/null and b/_resources/ace369b638966681b9558c42e25dd0b4.png differ diff --git a/_resources/ad9e649d35df7e77ce73056bbaf2cbb9.png b/_resources/ad9e649d35df7e77ce73056bbaf2cbb9.png new file mode 100644 index 0000000..7f21276 Binary files /dev/null and b/_resources/ad9e649d35df7e77ce73056bbaf2cbb9.png differ diff --git a/_resources/ae805ac0f184fdb6cf93d6b038af28a9.png b/_resources/ae805ac0f184fdb6cf93d6b038af28a9.png new file mode 100644 index 0000000..3331cf1 Binary files /dev/null and b/_resources/ae805ac0f184fdb6cf93d6b038af28a9.png differ diff --git a/_resources/alexnet.png b/_resources/alexnet.png new file mode 100644 index 0000000..1e258d3 Binary files /dev/null and b/_resources/alexnet.png differ diff --git a/_resources/architectureSparkApplication.png b/_resources/architectureSparkApplication.png new file mode 100644 index 0000000..8f11e15 Binary files /dev/null and b/_resources/architectureSparkApplication.png differ diff --git a/_resources/attention-alpha-e-nn.png b/_resources/attention-alpha-e-nn.png new file mode 100644 index 0000000..136a2fa Binary files /dev/null and b/_resources/attention-alpha-e-nn.png differ diff --git a/_resources/attention-alpha-vis.png b/_resources/attention-alpha-vis.png new file mode 100644 index 0000000..4b9850e Binary files /dev/null and b/_resources/attention-alpha-vis.png differ diff --git a/_resources/attention-alpha.svg b/_resources/attention-alpha.svg new file mode 100644 index 0000000..80188ef --- /dev/null +++ b/_resources/attention-alpha.svg @@ -0,0 +1,67 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='143.607927pt' height='37.119294pt' viewBox='-.239051 -.227883 143.607927 37.119294'> +<defs> +<path id='g6-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g6-61' d='M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z'/> +<path id='g0-0' d='M4.937484 13.736488C4.937484 13.688667 4.913574 13.664757 4.889664 13.628892C4.339726 13.043088 3.526775 12.07472 3.024658 10.126027C2.749689 9.038107 2.642092 7.806725 2.642092 6.694894C2.642092 3.550685 3.395268 1.350934 4.829888-.203238C4.937484-.310834 4.937484-.334745 4.937484-.358655C4.937484-.478207 4.841843-.478207 4.794022-.478207C4.614695-.478207 3.969116 .239103 3.813699 .418431C2.594271 1.865006 1.817186 4.016936 1.817186 6.682939C1.817186 8.380573 2.116065 10.783562 3.682192 12.803985C3.801743 12.947447 4.578829 13.85604 4.794022 13.85604C4.841843 13.85604 4.937484 13.85604 4.937484 13.736488Z'/> +<path id='g0-1' d='M3.646326 6.694894C3.646326 4.99726 3.347447 2.594271 1.78132 .573848C1.661768 .430386 .884682-.478207 .669489-.478207C.609714-.478207 .526027-.454296 .526027-.358655C.526027-.310834 .549938-.274969 .597758-.239103C1.171606 .382565 1.948692 1.350934 2.438854 3.251806C2.713823 4.339726 2.82142 5.571108 2.82142 6.682939C2.82142 7.890411 2.713823 9.109838 2.402989 10.281445C1.948692 11.955168 1.243337 12.911582 .633624 13.581071C.526027 13.688667 .526027 13.712578 .526027 13.736488C.526027 13.83213 .609714 13.85604 .669489 13.85604C.848817 13.85604 1.506351 13.126775 1.649813 12.959402C2.86924 11.512827 3.646326 9.360897 3.646326 6.694894Z'/> +<path id='g0-80' d='M5.033126 6.38406L.789041 11.632379C.6934 11.75193 .681445 11.775841 .681445 11.823661C.681445 11.955168 .789041 11.955168 1.004234 11.955168H10.915068L11.943213 8.978331H11.644334C11.345455 9.874969 10.544458 10.604234 9.528269 10.950934C9.336986 11.01071 8.51208 11.297634 6.75467 11.297634H1.673724L5.822167 6.168867C5.905853 6.06127 5.929763 6.025405 5.929763 5.977584S5.917808 5.917808 5.846077 5.810212L1.960648 .478207H6.694894C8.057783 .478207 10.807472 .561893 11.644334 2.797509H11.943213L10.915068 0H1.004234C.681445 0 .669489 .011955 .669489 .382565L5.033126 6.38406Z'/> +<path id='g3-120' d='M3.36538-2.349191C3.156164-2.289415 3.108344-2.116065 3.108344-2.026401C3.108344-1.835118 3.263761-1.793275 3.347447-1.793275C3.520797-1.793275 3.694147-1.936737 3.694147-2.169863C3.694147-2.492653 3.341469-2.636115 3.036613-2.636115C2.642092-2.636115 2.402989-2.331258 2.337235-2.217684C2.259527-2.367123 2.032379-2.636115 1.58406-2.636115C.896638-2.636115 .490162-1.924782 .490162-1.715567C.490162-1.685679 .514072-1.63188 .597758-1.63188S.699377-1.667746 .71731-1.721544C.86675-2.205729 1.273225-2.438854 1.566127-2.438854S1.948692-2.247572 1.948692-2.050311C1.948692-1.97858 1.948692-1.924782 1.900872-1.739477C1.763387-1.183562 1.63188-.639601 1.601993-.56787C1.512329-.340722 1.297136-.137484 1.046077-.137484C1.010212-.137484 .842839-.137484 .705355-.227148C.938481-.304857 .962391-.502117 .962391-.549938C.962391-.705355 .842839-.783064 .723288-.783064C.555915-.783064 .376588-.651557 .376588-.406476C.376588-.065753 .753176 .059776 1.034122 .059776C1.374844 .059776 1.619925-.17335 1.733499-.358655C1.853051-.107597 2.139975 .059776 2.480697 .059776C3.186052 .059776 3.580573-.663512 3.580573-.860772C3.580573-.872727 3.574595-.944458 3.466999-.944458C3.383313-.944458 3.371357-.902615 3.353425-.848817C3.180075-.328767 2.755666-.137484 2.504608-.137484C2.27746-.137484 2.116065-.268991 2.116065-.52005C2.116065-.633624 2.145953-.765131 2.199751-.974346L2.391034-1.751432C2.450809-1.984558 2.480697-2.092154 2.606227-2.235616C2.689913-2.32528 2.833375-2.438854 3.024658-2.438854C3.054545-2.438854 3.233873-2.438854 3.36538-2.349191Z'/> +<path id='g7-40' d='M3.88543 2.905106C3.88543 2.86924 3.88543 2.84533 3.682192 2.642092C2.486675 1.43462 1.817186-.537983 1.817186-2.976837C1.817186-5.296139 2.379078-7.292653 3.765878-8.703362C3.88543-8.810959 3.88543-8.834869 3.88543-8.870735C3.88543-8.942466 3.825654-8.966376 3.777833-8.966376C3.622416-8.966376 2.642092-8.105604 2.056289-6.933998C1.446575-5.726526 1.171606-4.447323 1.171606-2.976837C1.171606-1.912827 1.338979-.490162 1.960648 .789041C2.666002 2.223661 3.646326 3.000747 3.777833 3.000747C3.825654 3.000747 3.88543 2.976837 3.88543 2.905106Z'/> +<path id='g7-41' d='M3.371357-2.976837C3.371357-3.88543 3.251806-5.36787 2.582316-6.75467C1.876961-8.18929 .896638-8.966376 .765131-8.966376C.71731-8.966376 .657534-8.942466 .657534-8.870735C.657534-8.834869 .657534-8.810959 .860772-8.607721C2.056289-7.400249 2.725778-5.427646 2.725778-2.988792C2.725778-.669489 2.163885 1.327024 .777086 2.737733C.657534 2.84533 .657534 2.86924 .657534 2.905106C.657534 2.976837 .71731 3.000747 .765131 3.000747C.920548 3.000747 1.900872 2.139975 2.486675 .968369C3.096389-.251059 3.371357-1.542217 3.371357-2.976837Z'/> +<path id='g7-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g1-48' d='M1.876961-2.809465C1.918804-2.899128 1.93076-2.964882 1.93076-3.000747C1.93076-3.186052 1.763387-3.341469 1.566127-3.341469C1.327024-3.341469 1.255293-3.132254 1.231382-3.048568L.394521-.490162C.382565-.454296 .364633-.406476 .364633-.382565C.364633-.280946 .603736-.233126 .609714-.233126C.663512-.233126 .687422-.286924 .711333-.334745L1.876961-2.809465Z'/> +<path id='g4-59' d='M1.490411-.119552C1.490411 .398506 1.378829 .852802 .884682 1.346949C.852802 1.370859 .836862 1.3868 .836862 1.42665C.836862 1.490411 .900623 1.538232 .956413 1.538232C1.052055 1.538232 1.713574 .908593 1.713574-.02391C1.713574-.533998 1.522291-.884682 1.171606-.884682C.892653-.884682 .73325-.661519 .73325-.446326C.73325-.223163 .884682 0 1.179577 0C1.370859 0 1.490411-.111582 1.490411-.119552Z'/> +<path id='g4-84' d='M3.602491-4.821918C3.674222-5.108842 3.682192-5.124782 4.008966-5.124782H4.614695C5.443587-5.124782 5.539228-4.861768 5.539228-4.463263C5.539228-4.26401 5.491407-3.921295 5.483437-3.881445C5.467497-3.793773 5.459527-3.722042 5.459527-3.706102C5.459527-3.602491 5.531258-3.57858 5.579078-3.57858C5.66675-3.57858 5.69863-3.626401 5.72254-3.777833L5.937733-5.276214C5.937733-5.387796 5.842092-5.387796 5.69863-5.387796H1.004234C.804981-5.387796 .789041-5.387796 .73325-5.220423L.247073-3.841594C.231133-3.801743 .207223-3.737983 .207223-3.690162C.207223-3.626401 .263014-3.57858 .326775-3.57858C.414446-3.57858 .430386-3.618431 .478207-3.753923C.932503-5.029141 1.163636-5.124782 2.375093-5.124782H2.685928C2.925031-5.124782 2.933001-5.116812 2.933001-5.053051C2.933001-5.029141 2.901121-4.869738 2.893151-4.837858L1.841096-.653549C1.769365-.350685 1.745455-.263014 .916563-.263014C.661519-.263014 .581818-.263014 .581818-.111582C.581818-.103611 .581818 0 .71731 0C.932503 0 1.482441-.02391 1.697634-.02391H2.375093C2.598257-.02391 3.156164 0 3.379328 0C3.443088 0 3.56264 0 3.56264-.151432C3.56264-.263014 3.474969-.263014 3.259776-.263014C3.068493-.263014 3.004732-.263014 2.797509-.278954C2.542466-.302864 2.510585-.334745 2.510585-.438356C2.510585-.470237 2.518555-.502117 2.542466-.581818L3.602491-4.821918Z'/> +<path id='g4-116' d='M1.761395-3.172105H2.542466C2.693898-3.172105 2.789539-3.172105 2.789539-3.323537C2.789539-3.435118 2.685928-3.435118 2.550436-3.435118H1.825156L2.11208-4.566874C2.14396-4.686426 2.14396-4.726276 2.14396-4.734247C2.14396-4.901619 2.016438-4.98132 1.880946-4.98132C1.609963-4.98132 1.554172-4.766127 1.466501-4.407472L1.219427-3.435118H.454296C.302864-3.435118 .199253-3.435118 .199253-3.283686C.199253-3.172105 .302864-3.172105 .438356-3.172105H1.155666L.67746-1.259278C.629639-1.060025 .557908-.781071 .557908-.669489C.557908-.191283 .948443 .079701 1.370859 .079701C2.223661 .079701 2.709838-1.044085 2.709838-1.139726C2.709838-1.227397 2.638107-1.243337 2.590286-1.243337C2.502615-1.243337 2.494645-1.211457 2.438854-1.091905C2.279452-.70934 1.880946-.143462 1.39477-.143462C1.227397-.143462 1.131756-.255044 1.131756-.518057C1.131756-.669489 1.155666-.757161 1.179577-.860772L1.761395-3.172105Z'/> +<path id='g2-104' d='M2.765629-5.642839C2.81345-5.746451 2.81345-5.762391 2.81345-5.794271C2.81345-5.897883 2.733748-5.977584 2.630137-5.977584C2.510585-5.977584 2.470735-5.889913 2.438854-5.802242L.980324-2.14396C.932503-2.040349 .932503-2.024408 .932503-1.992528S.932503-1.944707 .980324-1.841096L2.422914 1.793275C2.478705 1.928767 2.526526 1.992528 2.630137 1.992528S2.81345 1.912827 2.81345 1.809215C2.81345 1.777335 2.81345 1.761395 2.765629 1.657783L1.315068-1.992528L2.765629-5.642839Z'/> +<path id='g2-105' d='M2.303362-1.841096C2.351183-1.944707 2.351183-1.960648 2.351183-1.992528S2.351183-2.040349 2.303362-2.14396L.860772-5.778331C.804981-5.913823 .757161-5.977584 .653549-5.977584S.470237-5.897883 .470237-5.794271C.470237-5.762391 .470237-5.746451 .518057-5.642839L1.968618-1.992528L.518057 1.657783C.470237 1.761395 .470237 1.777335 .470237 1.809215C.470237 1.912827 .549938 1.992528 .653549 1.992528C.773101 1.992528 .812951 1.904857 .844832 1.817186L2.303362-1.841096Z'/> +<path id='g5-11' d='M5.535243-3.024658C5.535243-4.184309 4.877709-5.272229 3.610461-5.272229C2.044334-5.272229 .478207-3.56264 .478207-1.865006C.478207-.824907 1.123786 .119552 2.343213 .119552C3.084433 .119552 3.969116-.167372 4.817933-.884682C4.985305-.215193 5.355915 .119552 5.869988 .119552C6.515567 .119552 6.838356-.549938 6.838356-.705355C6.838356-.812951 6.75467-.812951 6.718804-.812951C6.623163-.812951 6.611208-.777086 6.575342-.681445C6.467746-.382565 6.192777-.119552 5.905853-.119552C5.535243-.119552 5.535243-.884682 5.535243-1.613948C6.75467-3.072478 7.041594-4.578829 7.041594-4.590785C7.041594-4.698381 6.945953-4.698381 6.910087-4.698381C6.802491-4.698381 6.790535-4.662516 6.742715-4.447323C6.587298-3.921295 6.276463-2.988792 5.535243-2.008468V-3.024658ZM4.782067-1.171606C3.730012-.227148 2.785554-.119552 2.367123-.119552C1.518306-.119552 1.279203-.872727 1.279203-1.43462C1.279203-1.948692 1.542217-3.16812 1.912827-3.825654C2.402989-4.662516 3.072478-5.033126 3.610461-5.033126C4.770112-5.033126 4.770112-3.514819 4.770112-2.510585C4.770112-2.211706 4.758157-1.900872 4.758157-1.601993C4.758157-1.362889 4.770112-1.303113 4.782067-1.171606Z'/> +<path id='g5-101' d='M2.139975-2.773599C2.462765-2.773599 3.275716-2.797509 3.849564-3.012702C4.758157-3.359402 4.841843-4.052802 4.841843-4.267995C4.841843-4.794022 4.387547-5.272229 3.598506-5.272229C2.343213-5.272229 .537983-4.136488 .537983-2.008468C.537983-.753176 1.255293 .119552 2.343213 .119552C3.969116 .119552 4.99726-1.147696 4.99726-1.303113C4.99726-1.374844 4.925529-1.43462 4.877709-1.43462C4.841843-1.43462 4.829888-1.422665 4.722291-1.315068C3.957161-.298879 2.82142-.119552 2.367123-.119552C1.685679-.119552 1.327024-.657534 1.327024-1.542217C1.327024-1.709589 1.327024-2.008468 1.506351-2.773599H2.139975ZM1.566127-3.012702C2.080199-4.853798 3.21594-5.033126 3.598506-5.033126C4.124533-5.033126 4.483188-4.722291 4.483188-4.267995C4.483188-3.012702 2.570361-3.012702 2.068244-3.012702H1.566127Z'/> +<path id='g5-112' d='M.514072 1.518306C.430386 1.876961 .382565 1.972603-.107597 1.972603C-.251059 1.972603-.37061 1.972603-.37061 2.199751C-.37061 2.223661-.358655 2.319303-.227148 2.319303C-.071731 2.319303 .095641 2.295392 .251059 2.295392H.765131C1.016189 2.295392 1.625903 2.319303 1.876961 2.319303C1.948692 2.319303 2.092154 2.319303 2.092154 2.10411C2.092154 1.972603 2.008468 1.972603 1.80523 1.972603C1.255293 1.972603 1.219427 1.888917 1.219427 1.793275C1.219427 1.649813 1.75741-.406476 1.829141-.681445C1.960648-.3467 2.283437 .119552 2.905106 .119552C4.25604 .119552 5.71457-1.637858 5.71457-3.395268C5.71457-4.495143 5.092902-5.272229 4.196264-5.272229C3.431133-5.272229 2.785554-4.531009 2.654047-4.363636C2.558406-4.961395 2.092154-5.272229 1.613948-5.272229C1.267248-5.272229 .992279-5.104857 .765131-4.65056C.549938-4.220174 .382565-3.490909 .382565-3.443088S.430386-3.335492 .514072-3.335492C.609714-3.335492 .621669-3.347447 .6934-3.622416C.872727-4.327771 1.099875-5.033126 1.578082-5.033126C1.853051-5.033126 1.948692-4.841843 1.948692-4.483188C1.948692-4.196264 1.912827-4.076712 1.865006-3.861519L.514072 1.518306ZM2.582316-3.730012C2.666002-4.064757 3.000747-4.411457 3.19203-4.578829C3.323537-4.698381 3.718057-5.033126 4.172354-5.033126C4.698381-5.033126 4.937484-4.507098 4.937484-3.88543C4.937484-3.311582 4.60274-1.960648 4.303861-1.338979C4.004981-.6934 3.455044-.119552 2.905106-.119552C2.092154-.119552 1.960648-1.147696 1.960648-1.195517C1.960648-1.231382 1.984558-1.327024 1.996513-1.3868L2.582316-3.730012Z'/> +<path id='g5-120' d='M5.66675-4.877709C5.284184-4.805978 5.140722-4.519054 5.140722-4.291905C5.140722-4.004981 5.36787-3.90934 5.535243-3.90934C5.893898-3.90934 6.144956-4.220174 6.144956-4.542964C6.144956-5.045081 5.571108-5.272229 5.068991-5.272229C4.339726-5.272229 3.93325-4.554919 3.825654-4.327771C3.550685-5.224408 2.809465-5.272229 2.594271-5.272229C1.374844-5.272229 .729265-3.706102 .729265-3.443088C.729265-3.395268 .777086-3.335492 .860772-3.335492C.956413-3.335492 .980324-3.407223 1.004234-3.455044C1.41071-4.782067 2.211706-5.033126 2.558406-5.033126C3.096389-5.033126 3.203985-4.531009 3.203985-4.244085C3.203985-3.981071 3.132254-3.706102 2.988792-3.132254L2.582316-1.494396C2.402989-.777086 2.056289-.119552 1.422665-.119552C1.362889-.119552 1.06401-.119552 .812951-.274969C1.243337-.358655 1.338979-.71731 1.338979-.860772C1.338979-1.099875 1.159651-1.243337 .932503-1.243337C.645579-1.243337 .334745-.992279 .334745-.609714C.334745-.107597 .896638 .119552 1.41071 .119552C1.984558 .119552 2.391034-.334745 2.642092-.824907C2.833375-.119552 3.431133 .119552 3.873474 .119552C5.092902 .119552 5.738481-1.446575 5.738481-1.709589C5.738481-1.769365 5.69066-1.817186 5.618929-1.817186C5.511333-1.817186 5.499377-1.75741 5.463512-1.661768C5.140722-.609714 4.447323-.119552 3.90934-.119552C3.490909-.119552 3.263761-.430386 3.263761-.920548C3.263761-1.183562 3.311582-1.374844 3.502864-2.163885L3.921295-3.789788C4.100623-4.507098 4.507098-5.033126 5.057036-5.033126C5.080946-5.033126 5.415691-5.033126 5.66675-4.877709Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -60.996658)'> +<use x='56.413267' y='73.117198' xlink:href='#g5-11'/> +<use x='63.934991' y='68.181012' xlink:href='#g2-104'/> +<use x='67.228245' y='68.181012' xlink:href='#g4-116'/> +<use x='70.286266' y='68.181012' xlink:href='#g4-59'/> +<use x='72.638589' y='68.181012' xlink:href='#g4-116'/> +<use x='75.69661' y='65.368227' xlink:href='#g1-48'/> +<use x='78.380987' y='68.181012' xlink:href='#g2-105'/> +<use x='85.493201' y='73.117198' xlink:href='#g7-61'/> +<use x='114.023587' y='64.270369' xlink:href='#g5-101'/> +<use x='119.449027' y='64.270369' xlink:href='#g5-120'/> +<use x='126.101115' y='64.270369' xlink:href='#g5-112'/> +<use x='133.968755' y='54.586606' xlink:href='#g0-0'/> +<use x='139.448238' y='64.270369' xlink:href='#g5-101'/> +<use x='144.873678' y='59.931932' xlink:href='#g2-104'/> +<use x='148.166931' y='59.931932' xlink:href='#g4-116'/> +<use x='151.224952' y='59.931932' xlink:href='#g4-59'/> +<use x='153.577276' y='59.931932' xlink:href='#g4-116'/> +<use x='156.635297' y='57.119147' xlink:href='#g1-48'/> +<use x='159.319673' y='59.931932' xlink:href='#g2-105'/> +<use x='163.111058' y='54.586606' xlink:href='#g0-1'/> +<rect x='99.114196' y='69.889313' height='.478187' width='84.385732'/> +<use x='99.114196' y='74.093457' xlink:href='#g0-80'/> +<use x='111.733576' y='77.248298' xlink:href='#g4-84'/> +<use x='116.663772' y='78.244562' xlink:href='#g3-120'/> +<use x='111.733576' y='86.546909' xlink:href='#g4-116'/> +<use x='114.791596' y='84.277648' xlink:href='#g1-48'/> +<use x='117.475973' y='86.546909' xlink:href='#g6-61'/> +<use x='124.062479' y='86.546909' xlink:href='#g6-49'/> +<use x='130.787292' y='83.059909' xlink:href='#g5-101'/> +<use x='136.212732' y='83.059909' xlink:href='#g5-120'/> +<use x='142.864819' y='83.059909' xlink:href='#g5-112'/> +<use x='150.73246' y='83.059909' xlink:href='#g7-40'/> +<use x='155.284786' y='83.059909' xlink:href='#g5-101'/> +<use x='160.710226' y='79.606201' xlink:href='#g2-104'/> +<use x='164.003479' y='79.606201' xlink:href='#g4-116'/> +<use x='167.0615' y='79.606201' xlink:href='#g4-59'/> +<use x='169.413824' y='79.606201' xlink:href='#g4-116'/> +<use x='172.471845' y='77.33694' xlink:href='#g1-48'/> +<use x='175.156221' y='79.606201' xlink:href='#g2-105'/> +<use x='178.947606' y='83.059909' xlink:href='#g7-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/attention-intuition.png b/_resources/attention-intuition.png new file mode 100644 index 0000000..2e0ae95 Binary files /dev/null and b/_resources/attention-intuition.png differ diff --git a/_resources/attention-model.png b/_resources/attention-model.png new file mode 100644 index 0000000..2faeda2 Binary files /dev/null and b/_resources/attention-model.png differ diff --git a/_resources/b138f59fcc154b98b815a2dc79b89003.png b/_resources/b138f59fcc154b98b815a2dc79b89003.png new file mode 100644 index 0000000..79da0ad Binary files /dev/null and b/_resources/b138f59fcc154b98b815a2dc79b89003.png differ diff --git a/_resources/b190151ee52f3e4c4e48f19afe65db7a.png b/_resources/b190151ee52f3e4c4e48f19afe65db7a.png new file mode 100644 index 0000000..3321343 Binary files /dev/null and b/_resources/b190151ee52f3e4c4e48f19afe65db7a.png differ diff --git a/_resources/b3fabbeba3f22c574c31078c58c5dbe8.png b/_resources/b3fabbeba3f22c574c31078c58c5dbe8.png new file mode 100644 index 0000000..e0e892a Binary files /dev/null and b/_resources/b3fabbeba3f22c574c31078c58c5dbe8.png differ diff --git a/_resources/b4a66850e6f618c5b36f61b9efd52678.png b/_resources/b4a66850e6f618c5b36f61b9efd52678.png new file mode 100644 index 0000000..f65c764 Binary files /dev/null and b/_resources/b4a66850e6f618c5b36f61b9efd52678.png differ diff --git a/_resources/b55dc1138ff9d80431c97102da97c98c.png b/_resources/b55dc1138ff9d80431c97102da97c98c.png new file mode 100644 index 0000000..a48a6ae Binary files /dev/null and b/_resources/b55dc1138ff9d80431c97102da97c98c.png differ diff --git a/_resources/b685f0b119c20427c202a7143b227b60.png b/_resources/b685f0b119c20427c202a7143b227b60.png new file mode 100644 index 0000000..ac77fd3 Binary files /dev/null and b/_resources/b685f0b119c20427c202a7143b227b60.png differ diff --git a/_resources/b7f8b2f785a9637ea5a22abe2877bca5.png b/_resources/b7f8b2f785a9637ea5a22abe2877bca5.png new file mode 100644 index 0000000..19deb36 Binary files /dev/null and b/_resources/b7f8b2f785a9637ea5a22abe2877bca5.png differ diff --git a/_resources/b97bd5343f507eb8529bc2c00d30084d.png b/_resources/b97bd5343f507eb8529bc2c00d30084d.png new file mode 100644 index 0000000..f244fd4 Binary files /dev/null and b/_resources/b97bd5343f507eb8529bc2c00d30084d.png differ diff --git a/_resources/backprop_flow.png b/_resources/backprop_flow.png new file mode 100644 index 0000000..11da5c4 Binary files /dev/null and b/_resources/backprop_flow.png differ diff --git a/_resources/batch-norm-nn.png b/_resources/batch-norm-nn.png new file mode 100644 index 0000000..098b6bd Binary files /dev/null and b/_resources/batch-norm-nn.png differ diff --git a/_resources/batch-norm.png b/_resources/batch-norm.png new file mode 100644 index 0000000..0df1246 Binary files /dev/null and b/_resources/batch-norm.png differ diff --git a/_resources/be8b17237548f72ecd8013f80df036dc.png b/_resources/be8b17237548f72ecd8013f80df036dc.png new file mode 100644 index 0000000..c8aa793 Binary files /dev/null and b/_resources/be8b17237548f72ecd8013f80df036dc.png differ diff --git a/_resources/beam-search-1.png b/_resources/beam-search-1.png new file mode 100644 index 0000000..d70ac82 Binary files /dev/null and b/_resources/beam-search-1.png differ diff --git a/_resources/beam-search-2.png b/_resources/beam-search-2.png new file mode 100644 index 0000000..f20e468 Binary files /dev/null and b/_resources/beam-search-2.png differ diff --git a/_resources/beam-search-logp.svg b/_resources/beam-search-logp.svg new file mode 100644 index 0000000..a46aa91 --- /dev/null +++ b/_resources/beam-search-logp.svg @@ -0,0 +1,74 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='237.01496pt' height='39.839208pt' viewBox='-.239051 -.233284 237.01496 39.839208'> +<defs> +<path id='g1-0' d='M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z'/> +<path id='g2-106' d='M1.900872-8.53599C1.900872-8.751183 1.900872-8.966376 1.661768-8.966376S1.422665-8.751183 1.422665-8.53599V2.558406C1.422665 2.773599 1.422665 2.988792 1.661768 2.988792S1.900872 2.773599 1.900872 2.558406V-8.53599Z'/> +<path id='g6-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g6-61' d='M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z'/> +<path id='g0-88' d='M15.135243 16.737235L16.581818 12.911582H16.282939C15.816687 14.154919 14.54944 14.96787 13.174595 15.326526C12.923537 15.386301 11.75193 15.697136 9.456538 15.697136H2.247572L8.332752 8.5599C8.416438 8.464259 8.440349 8.428394 8.440349 8.368618C8.440349 8.344707 8.440349 8.308842 8.356663 8.18929L2.785554 .573848H9.336986C10.938979 .573848 12.026899 .74122 12.134496 .765131C12.780075 .860772 13.820174 1.06401 14.764633 1.661768C15.063512 1.853051 15.876463 2.391034 16.282939 3.359402H16.581818L15.135243 0H1.004234C.729265 0 .71731 .011955 .681445 .083686C.669489 .119552 .669489 .3467 .669489 .478207L6.993773 9.133748L.800996 16.390535C.681445 16.533998 .681445 16.593773 .681445 16.605729C.681445 16.737235 .789041 16.737235 1.004234 16.737235H15.135243Z'/> +<path id='g3-121' d='M3.56264-2.295392C3.574595-2.337235 3.580573-2.361146 3.580573-2.379078C3.580573-2.450809 3.526775-2.576339 3.36538-2.576339C3.263761-2.576339 3.162142-2.510585 3.120299-2.426899C3.096389-2.385056 3.054545-2.199751 3.024658-2.086177L2.899128-1.590037L2.713823-.842839C2.67198-.675467 2.64807-.579826 2.408966-.352677C2.343213-.292902 2.15193-.137484 1.894894-.137484C1.452553-.137484 1.452553-.526027 1.452553-.633624C1.452553-.896638 1.524284-1.165629 1.787298-1.823163C1.847073-1.966625 1.865006-2.008468 1.865006-2.116065C1.865006-2.450809 1.566127-2.636115 1.267248-2.636115C.657534-2.636115 .364633-1.847073 .364633-1.715567C.364633-1.685679 .388543-1.63188 .472229-1.63188S.573848-1.667746 .591781-1.721544C.759153-2.30137 1.075965-2.438854 1.243337-2.438854C1.362889-2.438854 1.404732-2.361146 1.404732-2.223661C1.404732-2.098132 1.327024-1.894894 1.26127-1.733499C1.052055-1.207472 .980324-.932503 .980324-.71731C.980324-.14944 1.404732 .059776 1.870984 .059776C1.95467 .059776 2.235616 .059776 2.552428-.215193C2.47472 .113574 2.420922 .3467 2.133998 .657534C2.026401 .777086 1.75741 1.022167 1.398755 1.022167C1.344956 1.022167 1.08792 1.016189 .950436 .860772C1.21345 .812951 1.249315 .579826 1.249315 .526027C1.249315 .364633 1.117808 .298879 1.004234 .298879C.854795 .298879 .657534 .418431 .657534 .6934C.657534 .980324 .938481 1.219427 1.404732 1.219427C2.038356 1.219427 2.803487 .753176 2.988792 .005978L3.56264-2.295392Z'/> +<path id='g5-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/> +<path id='g5-59' d='M2.331258 .047821C2.331258-.645579 2.10411-1.159651 1.613948-1.159651C1.231382-1.159651 1.0401-.848817 1.0401-.585803S1.219427 0 1.625903 0C1.78132 0 1.912827-.047821 2.020423-.155417C2.044334-.179328 2.056289-.179328 2.068244-.179328C2.092154-.179328 2.092154-.011955 2.092154 .047821C2.092154 .442341 2.020423 1.219427 1.327024 1.996513C1.195517 2.139975 1.195517 2.163885 1.195517 2.187796C1.195517 2.247572 1.255293 2.307347 1.315068 2.307347C1.41071 2.307347 2.331258 1.422665 2.331258 .047821Z'/> +<path id='g5-112' d='M.514072 1.518306C.430386 1.876961 .382565 1.972603-.107597 1.972603C-.251059 1.972603-.37061 1.972603-.37061 2.199751C-.37061 2.223661-.358655 2.319303-.227148 2.319303C-.071731 2.319303 .095641 2.295392 .251059 2.295392H.765131C1.016189 2.295392 1.625903 2.319303 1.876961 2.319303C1.948692 2.319303 2.092154 2.319303 2.092154 2.10411C2.092154 1.972603 2.008468 1.972603 1.80523 1.972603C1.255293 1.972603 1.219427 1.888917 1.219427 1.793275C1.219427 1.649813 1.75741-.406476 1.829141-.681445C1.960648-.3467 2.283437 .119552 2.905106 .119552C4.25604 .119552 5.71457-1.637858 5.71457-3.395268C5.71457-4.495143 5.092902-5.272229 4.196264-5.272229C3.431133-5.272229 2.785554-4.531009 2.654047-4.363636C2.558406-4.961395 2.092154-5.272229 1.613948-5.272229C1.267248-5.272229 .992279-5.104857 .765131-4.65056C.549938-4.220174 .382565-3.490909 .382565-3.443088S.430386-3.335492 .514072-3.335492C.609714-3.335492 .621669-3.347447 .6934-3.622416C.872727-4.327771 1.099875-5.033126 1.578082-5.033126C1.853051-5.033126 1.948692-4.841843 1.948692-4.483188C1.948692-4.196264 1.912827-4.076712 1.865006-3.861519L.514072 1.518306ZM2.582316-3.730012C2.666002-4.064757 3.000747-4.411457 3.19203-4.578829C3.323537-4.698381 3.718057-5.033126 4.172354-5.033126C4.698381-5.033126 4.937484-4.507098 4.937484-3.88543C4.937484-3.311582 4.60274-1.960648 4.303861-1.338979C4.004981-.6934 3.455044-.119552 2.905106-.119552C2.092154-.119552 1.960648-1.147696 1.960648-1.195517C1.960648-1.231382 1.984558-1.327024 1.996513-1.3868L2.582316-3.730012Z'/> +<path id='g5-120' d='M5.66675-4.877709C5.284184-4.805978 5.140722-4.519054 5.140722-4.291905C5.140722-4.004981 5.36787-3.90934 5.535243-3.90934C5.893898-3.90934 6.144956-4.220174 6.144956-4.542964C6.144956-5.045081 5.571108-5.272229 5.068991-5.272229C4.339726-5.272229 3.93325-4.554919 3.825654-4.327771C3.550685-5.224408 2.809465-5.272229 2.594271-5.272229C1.374844-5.272229 .729265-3.706102 .729265-3.443088C.729265-3.395268 .777086-3.335492 .860772-3.335492C.956413-3.335492 .980324-3.407223 1.004234-3.455044C1.41071-4.782067 2.211706-5.033126 2.558406-5.033126C3.096389-5.033126 3.203985-4.531009 3.203985-4.244085C3.203985-3.981071 3.132254-3.706102 2.988792-3.132254L2.582316-1.494396C2.402989-.777086 2.056289-.119552 1.422665-.119552C1.362889-.119552 1.06401-.119552 .812951-.274969C1.243337-.358655 1.338979-.71731 1.338979-.860772C1.338979-1.099875 1.159651-1.243337 .932503-1.243337C.645579-1.243337 .334745-.992279 .334745-.609714C.334745-.107597 .896638 .119552 1.41071 .119552C1.984558 .119552 2.391034-.334745 2.642092-.824907C2.833375-.119552 3.431133 .119552 3.873474 .119552C5.092902 .119552 5.738481-1.446575 5.738481-1.709589C5.738481-1.769365 5.69066-1.817186 5.618929-1.817186C5.511333-1.817186 5.499377-1.75741 5.463512-1.661768C5.140722-.609714 4.447323-.119552 3.90934-.119552C3.490909-.119552 3.263761-.430386 3.263761-.920548C3.263761-1.183562 3.311582-1.374844 3.502864-2.163885L3.921295-3.789788C4.100623-4.507098 4.507098-5.033126 5.057036-5.033126C5.080946-5.033126 5.415691-5.033126 5.66675-4.877709Z'/> +<path id='g5-121' d='M3.144209 1.338979C2.82142 1.793275 2.355168 2.199751 1.769365 2.199751C1.625903 2.199751 1.052055 2.175841 .872727 1.625903C.908593 1.637858 .968369 1.637858 .992279 1.637858C1.350934 1.637858 1.590037 1.327024 1.590037 1.052055S1.362889 .681445 1.183562 .681445C.992279 .681445 .573848 .824907 .573848 1.41071C.573848 2.020423 1.08792 2.438854 1.769365 2.438854C2.964882 2.438854 4.172354 1.338979 4.507098 .011955L5.678705-4.65056C5.69066-4.710336 5.71457-4.782067 5.71457-4.853798C5.71457-5.033126 5.571108-5.152677 5.391781-5.152677C5.284184-5.152677 5.033126-5.104857 4.937484-4.746202L4.052802-1.231382C3.993026-1.016189 3.993026-.992279 3.897385-.860772C3.658281-.526027 3.263761-.119552 2.689913-.119552C2.020423-.119552 1.960648-.777086 1.960648-1.099875C1.960648-1.78132 2.283437-2.701868 2.606227-3.56264C2.737733-3.90934 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.033126 2.139975-4.638605C2.139975-4.327771 2.008468-3.981071 1.829141-3.526775C1.243337-1.960648 1.243337-1.566127 1.243337-1.279203C1.243337-.143462 2.056289 .119552 2.654047 .119552C3.000747 .119552 3.431133 .011955 3.849564-.430386L3.861519-.418431C3.682192 .286924 3.56264 .753176 3.144209 1.338979Z'/> +<path id='g4-60' d='M5.7066-4.112578C5.802242-4.160399 5.873973-4.208219 5.873973-4.311831S5.794271-4.495143 5.69066-4.495143C5.66675-4.495143 5.650809-4.495143 5.547198-4.439352L.868742-2.191781C.773101-2.14396 .70137-2.096139 .70137-1.992528S.773101-1.841096 .868742-1.793275L5.547198 .454296C5.650809 .510087 5.66675 .510087 5.69066 .510087C5.794271 .510087 5.873973 .430386 5.873973 .326775S5.802242 .175342 5.7066 .127522L1.307098-1.992528L5.7066-4.112578Z'/> +<path id='g4-62' d='M5.7066-1.793275C5.802242-1.841096 5.873973-1.888917 5.873973-1.992528S5.802242-2.14396 5.7066-2.191781L1.028144-4.439352C.924533-4.495143 .908593-4.495143 .884682-4.495143C.781071-4.495143 .70137-4.415442 .70137-4.311831S.773101-4.160399 .868742-4.112578L5.268244-1.992528L.868742 .127522C.773101 .175342 .70137 .223163 .70137 .326775S.781071 .510087 .884682 .510087C.908593 .510087 .924533 .510087 1.028144 .454296L5.7066-1.793275Z'/> +<path id='g4-84' d='M3.602491-4.821918C3.674222-5.108842 3.682192-5.124782 4.008966-5.124782H4.614695C5.443587-5.124782 5.539228-4.861768 5.539228-4.463263C5.539228-4.26401 5.491407-3.921295 5.483437-3.881445C5.467497-3.793773 5.459527-3.722042 5.459527-3.706102C5.459527-3.602491 5.531258-3.57858 5.579078-3.57858C5.66675-3.57858 5.69863-3.626401 5.72254-3.777833L5.937733-5.276214C5.937733-5.387796 5.842092-5.387796 5.69863-5.387796H1.004234C.804981-5.387796 .789041-5.387796 .73325-5.220423L.247073-3.841594C.231133-3.801743 .207223-3.737983 .207223-3.690162C.207223-3.626401 .263014-3.57858 .326775-3.57858C.414446-3.57858 .430386-3.618431 .478207-3.753923C.932503-5.029141 1.163636-5.124782 2.375093-5.124782H2.685928C2.925031-5.124782 2.933001-5.116812 2.933001-5.053051C2.933001-5.029141 2.901121-4.869738 2.893151-4.837858L1.841096-.653549C1.769365-.350685 1.745455-.263014 .916563-.263014C.661519-.263014 .581818-.263014 .581818-.111582C.581818-.103611 .581818 0 .71731 0C.932503 0 1.482441-.02391 1.697634-.02391H2.375093C2.598257-.02391 3.156164 0 3.379328 0C3.443088 0 3.56264 0 3.56264-.151432C3.56264-.263014 3.474969-.263014 3.259776-.263014C3.068493-.263014 3.004732-.263014 2.797509-.278954C2.542466-.302864 2.510585-.334745 2.510585-.438356C2.510585-.470237 2.518555-.502117 2.542466-.581818L3.602491-4.821918Z'/> +<path id='g4-116' d='M1.761395-3.172105H2.542466C2.693898-3.172105 2.789539-3.172105 2.789539-3.323537C2.789539-3.435118 2.685928-3.435118 2.550436-3.435118H1.825156L2.11208-4.566874C2.14396-4.686426 2.14396-4.726276 2.14396-4.734247C2.14396-4.901619 2.016438-4.98132 1.880946-4.98132C1.609963-4.98132 1.554172-4.766127 1.466501-4.407472L1.219427-3.435118H.454296C.302864-3.435118 .199253-3.435118 .199253-3.283686C.199253-3.172105 .302864-3.172105 .438356-3.172105H1.155666L.67746-1.259278C.629639-1.060025 .557908-.781071 .557908-.669489C.557908-.191283 .948443 .079701 1.370859 .079701C2.223661 .079701 2.709838-1.044085 2.709838-1.139726C2.709838-1.227397 2.638107-1.243337 2.590286-1.243337C2.502615-1.243337 2.494645-1.211457 2.438854-1.091905C2.279452-.70934 1.880946-.143462 1.39477-.143462C1.227397-.143462 1.131756-.255044 1.131756-.518057C1.131756-.669489 1.155666-.757161 1.179577-.860772L1.761395-3.172105Z'/> +<path id='g4-121' d='M4.128518-3.004732C4.160399-3.116314 4.160399-3.132254 4.160399-3.188045C4.160399-3.387298 4.000996-3.435118 3.905355-3.435118C3.865504-3.435118 3.682192-3.427148 3.57858-3.219925C3.56264-3.180075 3.490909-2.893151 3.451059-2.725778L2.972852-.812951C2.964882-.789041 2.622167-.143462 2.040349-.143462C1.649813-.143462 1.514321-.430386 1.514321-.789041C1.514321-1.251308 1.785305-1.960648 1.968618-2.422914C2.048319-2.622167 2.072229-2.693898 2.072229-2.83736C2.072229-3.275716 1.721544-3.514819 1.354919-3.514819C.565878-3.514819 .239103-2.391034 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.70137-3.012702 1.044085-3.291656 1.331009-3.291656C1.45056-3.291656 1.522291-3.211955 1.522291-3.028643C1.522291-2.86127 1.458531-2.677958 1.40274-2.534496C1.075965-1.689664 .940473-1.283188 .940473-.908593C.940473-.127522 1.530262 .079701 2.000498 .079701C2.375093 .079701 2.646077-.087671 2.83736-.270984C2.725778 .175342 2.646077 .486177 2.343213 .868742C2.080199 1.195517 1.761395 1.40274 1.40274 1.40274C1.267248 1.40274 .964384 1.378829 .804981 1.139726C1.227397 1.107846 1.259278 .749191 1.259278 .70137C1.259278 .510087 1.115816 .406476 .948443 .406476C.773101 .406476 .494147 .541968 .494147 .932503C.494147 1.307098 .836862 1.625903 1.40274 1.625903C2.215691 1.625903 3.132254 .972354 3.371357 .00797L4.128518-3.004732Z'/> +<path id='g7-40' d='M3.88543 2.905106C3.88543 2.86924 3.88543 2.84533 3.682192 2.642092C2.486675 1.43462 1.817186-.537983 1.817186-2.976837C1.817186-5.296139 2.379078-7.292653 3.765878-8.703362C3.88543-8.810959 3.88543-8.834869 3.88543-8.870735C3.88543-8.942466 3.825654-8.966376 3.777833-8.966376C3.622416-8.966376 2.642092-8.105604 2.056289-6.933998C1.446575-5.726526 1.171606-4.447323 1.171606-2.976837C1.171606-1.912827 1.338979-.490162 1.960648 .789041C2.666002 2.223661 3.646326 3.000747 3.777833 3.000747C3.825654 3.000747 3.88543 2.976837 3.88543 2.905106Z'/> +<path id='g7-41' d='M3.371357-2.976837C3.371357-3.88543 3.251806-5.36787 2.582316-6.75467C1.876961-8.18929 .896638-8.966376 .765131-8.966376C.71731-8.966376 .657534-8.942466 .657534-8.870735C.657534-8.834869 .657534-8.810959 .860772-8.607721C2.056289-7.400249 2.725778-5.427646 2.725778-2.988792C2.725778-.669489 2.163885 1.327024 .777086 2.737733C.657534 2.84533 .657534 2.86924 .657534 2.905106C.657534 2.976837 .71731 3.000747 .765131 3.000747C.920548 3.000747 1.900872 2.139975 2.486675 .968369C3.096389-.251059 3.371357-1.542217 3.371357-2.976837Z'/> +<path id='g7-97' d='M4.614695-3.19203C4.614695-3.837609 4.614695-4.315816 4.088667-4.782067C3.670237-5.164633 3.132254-5.332005 2.606227-5.332005C1.625903-5.332005 .872727-4.686426 .872727-3.90934C.872727-3.56264 1.099875-3.395268 1.374844-3.395268C1.661768-3.395268 1.865006-3.598506 1.865006-3.88543C1.865006-4.375592 1.43462-4.375592 1.255293-4.375592C1.530262-4.877709 2.10411-5.092902 2.582316-5.092902C3.132254-5.092902 3.837609-4.638605 3.837609-3.56264V-3.084433C1.43462-3.048568 .526027-2.044334 .526027-1.123786C.526027-.179328 1.625903 .119552 2.355168 .119552C3.144209 .119552 3.682192-.358655 3.90934-.932503C3.957161-.37061 4.327771 .059776 4.841843 .059776C5.092902 .059776 5.786301-.107597 5.786301-1.06401V-1.733499H5.523288V-1.06401C5.523288-.382565 5.236364-.286924 5.068991-.286924C4.614695-.286924 4.614695-.920548 4.614695-1.099875V-3.19203ZM3.837609-1.685679C3.837609-.514072 2.964882-.119552 2.450809-.119552C1.865006-.119552 1.374844-.549938 1.374844-1.123786C1.374844-2.701868 3.407223-2.84533 3.837609-2.86924V-1.685679Z'/> +<path id='g7-103' d='M1.422665-2.163885C1.984558-1.793275 2.462765-1.793275 2.594271-1.793275C3.670237-1.793275 4.471233-2.606227 4.471233-3.526775C4.471233-3.849564 4.375592-4.303861 3.993026-4.686426C4.459278-5.164633 5.021171-5.164633 5.080946-5.164633C5.128767-5.164633 5.188543-5.164633 5.236364-5.140722C5.116812-5.092902 5.057036-4.97335 5.057036-4.841843C5.057036-4.674471 5.176588-4.531009 5.36787-4.531009C5.463512-4.531009 5.678705-4.590785 5.678705-4.853798C5.678705-5.068991 5.511333-5.403736 5.092902-5.403736C4.471233-5.403736 4.004981-5.021171 3.837609-4.841843C3.478954-5.116812 3.060523-5.272229 2.606227-5.272229C1.530262-5.272229 .729265-4.459278 .729265-3.53873C.729265-2.857285 1.147696-2.414944 1.267248-2.307347C1.123786-2.12802 .908593-1.78132 .908593-1.315068C.908593-.621669 1.327024-.32279 1.422665-.263014C.872727-.107597 .32279 .32279 .32279 .944458C.32279 1.769365 1.446575 2.450809 2.917061 2.450809C4.339726 2.450809 5.523288 1.817186 5.523288 .920548C5.523288 .621669 5.439601-.083686 4.722291-.454296C4.112578-.765131 3.514819-.765131 2.486675-.765131C1.75741-.765131 1.673724-.765131 1.458531-.992279C1.338979-1.111831 1.231382-1.338979 1.231382-1.590037C1.231382-1.793275 1.303113-1.996513 1.422665-2.163885ZM2.606227-2.044334C1.554172-2.044334 1.554172-3.251806 1.554172-3.526775C1.554172-3.741968 1.554172-4.23213 1.75741-4.554919C1.984558-4.901619 2.343213-5.021171 2.594271-5.021171C3.646326-5.021171 3.646326-3.813699 3.646326-3.53873C3.646326-3.323537 3.646326-2.833375 3.443088-2.510585C3.21594-2.163885 2.857285-2.044334 2.606227-2.044334ZM2.929016 2.199751C1.78132 2.199751 .908593 1.613948 .908593 .932503C.908593 .836862 .932503 .37061 1.3868 .059776C1.649813-.107597 1.75741-.107597 2.594271-.107597C3.58655-.107597 4.937484-.107597 4.937484 .932503C4.937484 1.637858 4.028892 2.199751 2.929016 2.199751Z'/> +<path id='g7-108' d='M2.056289-8.296887L.394521-8.16538V-7.81868C1.207472-7.81868 1.303113-7.734994 1.303113-7.149191V-.884682C1.303113-.3467 1.171606-.3467 .394521-.3467V0C.729265-.02391 1.315068-.02391 1.673724-.02391S2.630137-.02391 2.964882 0V-.3467C2.199751-.3467 2.056289-.3467 2.056289-.884682V-8.296887Z'/> +<path id='g7-109' d='M8.571856-2.905106C8.571856-4.016936 8.571856-4.351681 8.296887-4.734247C7.950187-5.200498 7.388294-5.272229 6.981818-5.272229C5.989539-5.272229 5.487422-4.554919 5.296139-4.088667C5.128767-5.009215 4.483188-5.272229 3.730012-5.272229C2.570361-5.272229 2.116065-4.27995 2.020423-4.040847H2.008468V-5.272229L.382565-5.140722V-4.794022C1.195517-4.794022 1.291158-4.710336 1.291158-4.124533V-.884682C1.291158-.3467 1.159651-.3467 .382565-.3467V0C.6934-.02391 1.338979-.02391 1.673724-.02391C2.020423-.02391 2.666002-.02391 2.976837 0V-.3467C2.211706-.3467 2.068244-.3467 2.068244-.884682V-3.108344C2.068244-4.363636 2.893151-5.033126 3.634371-5.033126S4.542964-4.423412 4.542964-3.694147V-.884682C4.542964-.3467 4.411457-.3467 3.634371-.3467V0C3.945205-.02391 4.590785-.02391 4.925529-.02391C5.272229-.02391 5.917808-.02391 6.228643 0V-.3467C5.463512-.3467 5.32005-.3467 5.32005-.884682V-3.108344C5.32005-4.363636 6.144956-5.033126 6.886177-5.033126S7.79477-4.423412 7.79477-3.694147V-.884682C7.79477-.3467 7.663263-.3467 6.886177-.3467V0C7.197011-.02391 7.84259-.02391 8.177335-.02391C8.524035-.02391 9.169614-.02391 9.480448 0V-.3467C8.88269-.3467 8.583811-.3467 8.571856-.705355V-2.905106Z'/> +<path id='g7-111' d='M5.487422-2.558406C5.487422-4.100623 4.315816-5.332005 2.929016-5.332005C1.494396-5.332005 .358655-4.064757 .358655-2.558406C.358655-1.028144 1.554172 .119552 2.917061 .119552C4.327771 .119552 5.487422-1.052055 5.487422-2.558406ZM2.929016-.143462C2.486675-.143462 1.948692-.334745 1.601993-.920548C1.279203-1.458531 1.267248-2.163885 1.267248-2.666002C1.267248-3.120299 1.267248-3.849564 1.637858-4.387547C1.972603-4.901619 2.49863-5.092902 2.917061-5.092902C3.383313-5.092902 3.88543-4.877709 4.208219-4.411457C4.578829-3.861519 4.578829-3.108344 4.578829-2.666002C4.578829-2.247572 4.578829-1.506351 4.267995-.944458C3.93325-.37061 3.383313-.143462 2.929016-.143462Z'/> +<path id='g7-114' d='M1.996513-2.785554C1.996513-3.945205 2.47472-5.033126 3.395268-5.033126C3.490909-5.033126 3.514819-5.033126 3.56264-5.021171C3.466999-4.97335 3.275716-4.901619 3.275716-4.578829C3.275716-4.23213 3.550685-4.100623 3.741968-4.100623C3.981071-4.100623 4.220174-4.25604 4.220174-4.578829C4.220174-4.937484 3.897385-5.272229 3.383313-5.272229C2.367123-5.272229 2.020423-4.172354 1.948692-3.945205H1.936737V-5.272229L.334745-5.140722V-4.794022C1.147696-4.794022 1.243337-4.710336 1.243337-4.124533V-.884682C1.243337-.3467 1.111831-.3467 .334745-.3467V0C.669489-.02391 1.327024-.02391 1.685679-.02391C2.008468-.02391 2.857285-.02391 3.132254 0V-.3467H2.893151C2.020423-.3467 1.996513-.478207 1.996513-.908593V-2.785554Z'/> +<path id='g7-120' d='M3.347447-2.82142C3.694147-3.275716 4.196264-3.921295 4.423412-4.172354C4.913574-4.722291 5.475467-4.805978 5.858032-4.805978V-5.152677C5.34396-5.128767 5.32005-5.128767 4.853798-5.128767C4.399502-5.128767 4.375592-5.128767 3.777833-5.152677V-4.805978C3.93325-4.782067 4.124533-4.710336 4.124533-4.435367C4.124533-4.23213 4.016936-4.100623 3.945205-4.004981L3.180075-3.036613L2.247572-4.267995C2.211706-4.315816 2.139975-4.423412 2.139975-4.507098C2.139975-4.578829 2.199751-4.794022 2.558406-4.805978V-5.152677C2.259527-5.128767 1.649813-5.128767 1.327024-5.128767C.932503-5.128767 .908593-5.128767 .179328-5.152677V-4.805978C.789041-4.805978 1.016189-4.782067 1.267248-4.459278L2.666002-2.630137C2.689913-2.606227 2.737733-2.534496 2.737733-2.49863S1.80523-1.291158 1.685679-1.135741C1.159651-.490162 .633624-.358655 .119552-.3467V0C.573848-.02391 .597758-.02391 1.111831-.02391C1.566127-.02391 1.590037-.02391 2.187796 0V-.3467C1.900872-.382565 1.853051-.561893 1.853051-.729265C1.853051-.920548 1.936737-1.016189 2.056289-1.171606C2.235616-1.422665 2.630137-1.912827 2.917061-2.283437L3.897385-1.004234C4.100623-.74122 4.100623-.71731 4.100623-.645579C4.100623-.549938 4.004981-.358655 3.682192-.3467V0C3.993026-.02391 4.578829-.02391 4.913574-.02391C5.308095-.02391 5.332005-.02391 6.049315 0V-.3467C5.415691-.3467 5.200498-.37061 4.913574-.753176L3.347447-2.82142Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.442282)'> +<use x='56.413267' y='76.479924' xlink:href='#g7-97'/> +<use x='62.266257' y='76.479924' xlink:href='#g7-114'/> +<use x='66.818583' y='76.479924' xlink:href='#g7-103'/> +<use x='74.826654' y='76.479924' xlink:href='#g7-109'/> +<use x='84.581638' y='76.479924' xlink:href='#g7-97'/> +<use x='90.434628' y='76.479924' xlink:href='#g7-120'/> +<use x='83.483872' y='83.653022' xlink:href='#g4-121'/> +<use x='102.525989' y='60.440006' xlink:href='#g4-84'/> +<use x='107.456185' y='61.43627' xlink:href='#g3-121'/> +<use x='98.605282' y='65.122459' xlink:href='#g0-88'/> +<use x='100.300235' y='90.228446' xlink:href='#g4-116'/> +<use x='103.358256' y='90.228446' xlink:href='#g6-61'/> +<use x='109.944763' y='90.228446' xlink:href='#g6-49'/> +<use x='117.866396' y='76.479924' xlink:href='#g7-108'/> +<use x='121.118058' y='76.479924' xlink:href='#g7-111'/> +<use x='126.971048' y='76.479924' xlink:href='#g7-103'/> +<use x='136.307451' y='76.479924' xlink:href='#g5-112'/> +<use x='142.182594' y='76.479924' xlink:href='#g7-40'/> +<use x='146.734919' y='76.479924' xlink:href='#g5-121'/> +<use x='152.871571' y='71.543738' xlink:href='#g4-60'/> +<use x='159.458078' y='71.543738' xlink:href='#g4-116'/> +<use x='162.516099' y='71.543738' xlink:href='#g4-62'/> +<use x='169.600737' y='76.479924' xlink:href='#g2-106'/> +<use x='172.921628' y='76.479924' xlink:href='#g5-120'/> +<use x='179.573715' y='76.479924' xlink:href='#g5-59'/> +<use x='184.817874' y='76.479924' xlink:href='#g5-121'/> +<use x='190.954525' y='71.543738' xlink:href='#g4-60'/> +<use x='197.541032' y='71.543738' xlink:href='#g6-49'/> +<use x='201.775215' y='71.543738' xlink:href='#g4-62'/> +<use x='208.859853' y='76.479924' xlink:href='#g5-59'/> +<use x='214.104012' y='76.479924' xlink:href='#g5-58'/> +<use x='217.355674' y='76.479924' xlink:href='#g5-58'/> +<use x='220.607335' y='76.479924' xlink:href='#g5-58'/> +<use x='223.858996' y='76.479924' xlink:href='#g5-59'/> +<use x='229.103155' y='76.479924' xlink:href='#g5-121'/> +<use x='235.239807' y='71.543738' xlink:href='#g4-60'/> +<use x='241.826313' y='71.543738' xlink:href='#g4-116'/> +<use x='244.884334' y='71.543738' xlink:href='#g1-0'/> +<use x='251.470841' y='71.543738' xlink:href='#g6-49'/> +<use x='255.705024' y='71.543738' xlink:href='#g4-62'/> +<use x='262.789662' y='76.479924' xlink:href='#g7-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/beam-search-norm.svg b/_resources/beam-search-norm.svg new file mode 100644 index 0000000..1ebfde2 --- /dev/null +++ b/_resources/beam-search-norm.svg @@ -0,0 +1,82 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='256.015837pt' height='39.839208pt' viewBox='-.239051 -.233284 256.015837 39.839208'> +<defs> +<path id='g1-0' d='M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z'/> +<path id='g2-106' d='M1.900872-8.53599C1.900872-8.751183 1.900872-8.966376 1.661768-8.966376S1.422665-8.751183 1.422665-8.53599V2.558406C1.422665 2.773599 1.422665 2.988792 1.661768 2.988792S1.900872 2.773599 1.900872 2.558406V-8.53599Z'/> +<path id='g6-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g6-61' d='M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z'/> +<path id='g0-88' d='M15.135243 16.737235L16.581818 12.911582H16.282939C15.816687 14.154919 14.54944 14.96787 13.174595 15.326526C12.923537 15.386301 11.75193 15.697136 9.456538 15.697136H2.247572L8.332752 8.5599C8.416438 8.464259 8.440349 8.428394 8.440349 8.368618C8.440349 8.344707 8.440349 8.308842 8.356663 8.18929L2.785554 .573848H9.336986C10.938979 .573848 12.026899 .74122 12.134496 .765131C12.780075 .860772 13.820174 1.06401 14.764633 1.661768C15.063512 1.853051 15.876463 2.391034 16.282939 3.359402H16.581818L15.135243 0H1.004234C.729265 0 .71731 .011955 .681445 .083686C.669489 .119552 .669489 .3467 .669489 .478207L6.993773 9.133748L.800996 16.390535C.681445 16.533998 .681445 16.593773 .681445 16.605729C.681445 16.737235 .789041 16.737235 1.004234 16.737235H15.135243Z'/> +<path id='g3-121' d='M3.56264-2.295392C3.574595-2.337235 3.580573-2.361146 3.580573-2.379078C3.580573-2.450809 3.526775-2.576339 3.36538-2.576339C3.263761-2.576339 3.162142-2.510585 3.120299-2.426899C3.096389-2.385056 3.054545-2.199751 3.024658-2.086177L2.899128-1.590037L2.713823-.842839C2.67198-.675467 2.64807-.579826 2.408966-.352677C2.343213-.292902 2.15193-.137484 1.894894-.137484C1.452553-.137484 1.452553-.526027 1.452553-.633624C1.452553-.896638 1.524284-1.165629 1.787298-1.823163C1.847073-1.966625 1.865006-2.008468 1.865006-2.116065C1.865006-2.450809 1.566127-2.636115 1.267248-2.636115C.657534-2.636115 .364633-1.847073 .364633-1.715567C.364633-1.685679 .388543-1.63188 .472229-1.63188S.573848-1.667746 .591781-1.721544C.759153-2.30137 1.075965-2.438854 1.243337-2.438854C1.362889-2.438854 1.404732-2.361146 1.404732-2.223661C1.404732-2.098132 1.327024-1.894894 1.26127-1.733499C1.052055-1.207472 .980324-.932503 .980324-.71731C.980324-.14944 1.404732 .059776 1.870984 .059776C1.95467 .059776 2.235616 .059776 2.552428-.215193C2.47472 .113574 2.420922 .3467 2.133998 .657534C2.026401 .777086 1.75741 1.022167 1.398755 1.022167C1.344956 1.022167 1.08792 1.016189 .950436 .860772C1.21345 .812951 1.249315 .579826 1.249315 .526027C1.249315 .364633 1.117808 .298879 1.004234 .298879C.854795 .298879 .657534 .418431 .657534 .6934C.657534 .980324 .938481 1.219427 1.404732 1.219427C2.038356 1.219427 2.803487 .753176 2.988792 .005978L3.56264-2.295392Z'/> +<path id='g5-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/> +<path id='g5-59' d='M2.331258 .047821C2.331258-.645579 2.10411-1.159651 1.613948-1.159651C1.231382-1.159651 1.0401-.848817 1.0401-.585803S1.219427 0 1.625903 0C1.78132 0 1.912827-.047821 2.020423-.155417C2.044334-.179328 2.056289-.179328 2.068244-.179328C2.092154-.179328 2.092154-.011955 2.092154 .047821C2.092154 .442341 2.020423 1.219427 1.327024 1.996513C1.195517 2.139975 1.195517 2.163885 1.195517 2.187796C1.195517 2.247572 1.255293 2.307347 1.315068 2.307347C1.41071 2.307347 2.331258 1.422665 2.331258 .047821Z'/> +<path id='g5-84' d='M4.985305-7.292653C5.057036-7.579577 5.080946-7.687173 5.260274-7.734994C5.355915-7.758904 5.750436-7.758904 6.001494-7.758904C7.197011-7.758904 7.758904-7.711083 7.758904-6.77858C7.758904-6.599253 7.711083-6.144956 7.639352-5.702615L7.627397-5.559153C7.627397-5.511333 7.675218-5.439601 7.746949-5.439601C7.866501-5.439601 7.866501-5.499377 7.902366-5.69066L8.249066-7.806725C8.272976-7.914321 8.272976-7.938232 8.272976-7.974097C8.272976-8.105604 8.201245-8.105604 7.962142-8.105604H1.422665C1.147696-8.105604 1.135741-8.093649 1.06401-7.878456L.334745-5.726526C.32279-5.702615 .286924-5.571108 .286924-5.559153C.286924-5.499377 .334745-5.439601 .406476-5.439601C.502117-5.439601 .526027-5.487422 .573848-5.642839C1.075965-7.089415 1.327024-7.758904 2.917061-7.758904H3.718057C4.004981-7.758904 4.124533-7.758904 4.124533-7.627397C4.124533-7.591532 4.124533-7.567621 4.064757-7.352428L2.462765-.932503C2.343213-.466252 2.319303-.3467 1.052055-.3467C.753176-.3467 .669489-.3467 .669489-.119552C.669489 0 .800996 0 .860772 0C1.159651 0 1.470486-.02391 1.769365-.02391H3.634371C3.93325-.02391 4.25604 0 4.554919 0C4.686426 0 4.805978 0 4.805978-.227148C4.805978-.3467 4.722291-.3467 4.411457-.3467C3.335492-.3467 3.335492-.454296 3.335492-.633624C3.335492-.645579 3.335492-.729265 3.383313-.920548L4.985305-7.292653Z'/> +<path id='g5-112' d='M.514072 1.518306C.430386 1.876961 .382565 1.972603-.107597 1.972603C-.251059 1.972603-.37061 1.972603-.37061 2.199751C-.37061 2.223661-.358655 2.319303-.227148 2.319303C-.071731 2.319303 .095641 2.295392 .251059 2.295392H.765131C1.016189 2.295392 1.625903 2.319303 1.876961 2.319303C1.948692 2.319303 2.092154 2.319303 2.092154 2.10411C2.092154 1.972603 2.008468 1.972603 1.80523 1.972603C1.255293 1.972603 1.219427 1.888917 1.219427 1.793275C1.219427 1.649813 1.75741-.406476 1.829141-.681445C1.960648-.3467 2.283437 .119552 2.905106 .119552C4.25604 .119552 5.71457-1.637858 5.71457-3.395268C5.71457-4.495143 5.092902-5.272229 4.196264-5.272229C3.431133-5.272229 2.785554-4.531009 2.654047-4.363636C2.558406-4.961395 2.092154-5.272229 1.613948-5.272229C1.267248-5.272229 .992279-5.104857 .765131-4.65056C.549938-4.220174 .382565-3.490909 .382565-3.443088S.430386-3.335492 .514072-3.335492C.609714-3.335492 .621669-3.347447 .6934-3.622416C.872727-4.327771 1.099875-5.033126 1.578082-5.033126C1.853051-5.033126 1.948692-4.841843 1.948692-4.483188C1.948692-4.196264 1.912827-4.076712 1.865006-3.861519L.514072 1.518306ZM2.582316-3.730012C2.666002-4.064757 3.000747-4.411457 3.19203-4.578829C3.323537-4.698381 3.718057-5.033126 4.172354-5.033126C4.698381-5.033126 4.937484-4.507098 4.937484-3.88543C4.937484-3.311582 4.60274-1.960648 4.303861-1.338979C4.004981-.6934 3.455044-.119552 2.905106-.119552C2.092154-.119552 1.960648-1.147696 1.960648-1.195517C1.960648-1.231382 1.984558-1.327024 1.996513-1.3868L2.582316-3.730012Z'/> +<path id='g5-120' d='M5.66675-4.877709C5.284184-4.805978 5.140722-4.519054 5.140722-4.291905C5.140722-4.004981 5.36787-3.90934 5.535243-3.90934C5.893898-3.90934 6.144956-4.220174 6.144956-4.542964C6.144956-5.045081 5.571108-5.272229 5.068991-5.272229C4.339726-5.272229 3.93325-4.554919 3.825654-4.327771C3.550685-5.224408 2.809465-5.272229 2.594271-5.272229C1.374844-5.272229 .729265-3.706102 .729265-3.443088C.729265-3.395268 .777086-3.335492 .860772-3.335492C.956413-3.335492 .980324-3.407223 1.004234-3.455044C1.41071-4.782067 2.211706-5.033126 2.558406-5.033126C3.096389-5.033126 3.203985-4.531009 3.203985-4.244085C3.203985-3.981071 3.132254-3.706102 2.988792-3.132254L2.582316-1.494396C2.402989-.777086 2.056289-.119552 1.422665-.119552C1.362889-.119552 1.06401-.119552 .812951-.274969C1.243337-.358655 1.338979-.71731 1.338979-.860772C1.338979-1.099875 1.159651-1.243337 .932503-1.243337C.645579-1.243337 .334745-.992279 .334745-.609714C.334745-.107597 .896638 .119552 1.41071 .119552C1.984558 .119552 2.391034-.334745 2.642092-.824907C2.833375-.119552 3.431133 .119552 3.873474 .119552C5.092902 .119552 5.738481-1.446575 5.738481-1.709589C5.738481-1.769365 5.69066-1.817186 5.618929-1.817186C5.511333-1.817186 5.499377-1.75741 5.463512-1.661768C5.140722-.609714 4.447323-.119552 3.90934-.119552C3.490909-.119552 3.263761-.430386 3.263761-.920548C3.263761-1.183562 3.311582-1.374844 3.502864-2.163885L3.921295-3.789788C4.100623-4.507098 4.507098-5.033126 5.057036-5.033126C5.080946-5.033126 5.415691-5.033126 5.66675-4.877709Z'/> +<path id='g5-121' d='M3.144209 1.338979C2.82142 1.793275 2.355168 2.199751 1.769365 2.199751C1.625903 2.199751 1.052055 2.175841 .872727 1.625903C.908593 1.637858 .968369 1.637858 .992279 1.637858C1.350934 1.637858 1.590037 1.327024 1.590037 1.052055S1.362889 .681445 1.183562 .681445C.992279 .681445 .573848 .824907 .573848 1.41071C.573848 2.020423 1.08792 2.438854 1.769365 2.438854C2.964882 2.438854 4.172354 1.338979 4.507098 .011955L5.678705-4.65056C5.69066-4.710336 5.71457-4.782067 5.71457-4.853798C5.71457-5.033126 5.571108-5.152677 5.391781-5.152677C5.284184-5.152677 5.033126-5.104857 4.937484-4.746202L4.052802-1.231382C3.993026-1.016189 3.993026-.992279 3.897385-.860772C3.658281-.526027 3.263761-.119552 2.689913-.119552C2.020423-.119552 1.960648-.777086 1.960648-1.099875C1.960648-1.78132 2.283437-2.701868 2.606227-3.56264C2.737733-3.90934 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.033126 2.139975-4.638605C2.139975-4.327771 2.008468-3.981071 1.829141-3.526775C1.243337-1.960648 1.243337-1.566127 1.243337-1.279203C1.243337-.143462 2.056289 .119552 2.654047 .119552C3.000747 .119552 3.431133 .011955 3.849564-.430386L3.861519-.418431C3.682192 .286924 3.56264 .753176 3.144209 1.338979Z'/> +<path id='g4-11' d='M4.064757-1.115816C4.805978-1.928767 5.068991-2.964882 5.068991-3.028643C5.068991-3.100374 5.021171-3.132254 4.94944-3.132254C4.845828-3.132254 4.837858-3.100374 4.790037-2.933001C4.566874-2.12005 4.088667-1.498381 4.064757-1.498381C4.048817-1.498381 4.048817-1.697634 4.048817-1.825156C4.032877-3.227895 3.124284-3.514819 2.582316-3.514819C1.458531-3.514819 .350685-2.422914 .350685-1.299128C.350685-.510087 .900623 .079701 1.745455 .079701C2.303362 .079701 2.893151-.119552 3.52279-.589788C3.698132 .039851 4.160399 .079701 4.303861 .079701C4.758157 .079701 5.021171-.326775 5.021171-.478207C5.021171-.573848 4.925529-.573848 4.901619-.573848C4.813948-.573848 4.798007-.549938 4.774097-.494147C4.646575-.159402 4.375592-.143462 4.335741-.143462C4.224159-.143462 4.096638-.143462 4.064757-1.115816ZM3.466999-.852802C2.901121-.342715 2.231631-.143462 1.769365-.143462C1.354919-.143462 .996264-.382565 .996264-1.020174C.996264-1.299128 1.123786-2.12005 1.498381-2.654047C1.817186-3.100374 2.247572-3.291656 2.574346-3.291656C3.012702-3.291656 3.259776-2.980822 3.363387-2.494645C3.482939-1.952677 3.419178-1.315068 3.466999-.852802Z'/> +<path id='g4-60' d='M5.7066-4.112578C5.802242-4.160399 5.873973-4.208219 5.873973-4.311831S5.794271-4.495143 5.69066-4.495143C5.66675-4.495143 5.650809-4.495143 5.547198-4.439352L.868742-2.191781C.773101-2.14396 .70137-2.096139 .70137-1.992528S.773101-1.841096 .868742-1.793275L5.547198 .454296C5.650809 .510087 5.66675 .510087 5.69066 .510087C5.794271 .510087 5.873973 .430386 5.873973 .326775S5.802242 .175342 5.7066 .127522L1.307098-1.992528L5.7066-4.112578Z'/> +<path id='g4-62' d='M5.7066-1.793275C5.802242-1.841096 5.873973-1.888917 5.873973-1.992528S5.802242-2.14396 5.7066-2.191781L1.028144-4.439352C.924533-4.495143 .908593-4.495143 .884682-4.495143C.781071-4.495143 .70137-4.415442 .70137-4.311831S.773101-4.160399 .868742-4.112578L5.268244-1.992528L.868742 .127522C.773101 .175342 .70137 .223163 .70137 .326775S.781071 .510087 .884682 .510087C.908593 .510087 .924533 .510087 1.028144 .454296L5.7066-1.793275Z'/> +<path id='g4-84' d='M3.602491-4.821918C3.674222-5.108842 3.682192-5.124782 4.008966-5.124782H4.614695C5.443587-5.124782 5.539228-4.861768 5.539228-4.463263C5.539228-4.26401 5.491407-3.921295 5.483437-3.881445C5.467497-3.793773 5.459527-3.722042 5.459527-3.706102C5.459527-3.602491 5.531258-3.57858 5.579078-3.57858C5.66675-3.57858 5.69863-3.626401 5.72254-3.777833L5.937733-5.276214C5.937733-5.387796 5.842092-5.387796 5.69863-5.387796H1.004234C.804981-5.387796 .789041-5.387796 .73325-5.220423L.247073-3.841594C.231133-3.801743 .207223-3.737983 .207223-3.690162C.207223-3.626401 .263014-3.57858 .326775-3.57858C.414446-3.57858 .430386-3.618431 .478207-3.753923C.932503-5.029141 1.163636-5.124782 2.375093-5.124782H2.685928C2.925031-5.124782 2.933001-5.116812 2.933001-5.053051C2.933001-5.029141 2.901121-4.869738 2.893151-4.837858L1.841096-.653549C1.769365-.350685 1.745455-.263014 .916563-.263014C.661519-.263014 .581818-.263014 .581818-.111582C.581818-.103611 .581818 0 .71731 0C.932503 0 1.482441-.02391 1.697634-.02391H2.375093C2.598257-.02391 3.156164 0 3.379328 0C3.443088 0 3.56264 0 3.56264-.151432C3.56264-.263014 3.474969-.263014 3.259776-.263014C3.068493-.263014 3.004732-.263014 2.797509-.278954C2.542466-.302864 2.510585-.334745 2.510585-.438356C2.510585-.470237 2.518555-.502117 2.542466-.581818L3.602491-4.821918Z'/> +<path id='g4-116' d='M1.761395-3.172105H2.542466C2.693898-3.172105 2.789539-3.172105 2.789539-3.323537C2.789539-3.435118 2.685928-3.435118 2.550436-3.435118H1.825156L2.11208-4.566874C2.14396-4.686426 2.14396-4.726276 2.14396-4.734247C2.14396-4.901619 2.016438-4.98132 1.880946-4.98132C1.609963-4.98132 1.554172-4.766127 1.466501-4.407472L1.219427-3.435118H.454296C.302864-3.435118 .199253-3.435118 .199253-3.283686C.199253-3.172105 .302864-3.172105 .438356-3.172105H1.155666L.67746-1.259278C.629639-1.060025 .557908-.781071 .557908-.669489C.557908-.191283 .948443 .079701 1.370859 .079701C2.223661 .079701 2.709838-1.044085 2.709838-1.139726C2.709838-1.227397 2.638107-1.243337 2.590286-1.243337C2.502615-1.243337 2.494645-1.211457 2.438854-1.091905C2.279452-.70934 1.880946-.143462 1.39477-.143462C1.227397-.143462 1.131756-.255044 1.131756-.518057C1.131756-.669489 1.155666-.757161 1.179577-.860772L1.761395-3.172105Z'/> +<path id='g4-121' d='M4.128518-3.004732C4.160399-3.116314 4.160399-3.132254 4.160399-3.188045C4.160399-3.387298 4.000996-3.435118 3.905355-3.435118C3.865504-3.435118 3.682192-3.427148 3.57858-3.219925C3.56264-3.180075 3.490909-2.893151 3.451059-2.725778L2.972852-.812951C2.964882-.789041 2.622167-.143462 2.040349-.143462C1.649813-.143462 1.514321-.430386 1.514321-.789041C1.514321-1.251308 1.785305-1.960648 1.968618-2.422914C2.048319-2.622167 2.072229-2.693898 2.072229-2.83736C2.072229-3.275716 1.721544-3.514819 1.354919-3.514819C.565878-3.514819 .239103-2.391034 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.70137-3.012702 1.044085-3.291656 1.331009-3.291656C1.45056-3.291656 1.522291-3.211955 1.522291-3.028643C1.522291-2.86127 1.458531-2.677958 1.40274-2.534496C1.075965-1.689664 .940473-1.283188 .940473-.908593C.940473-.127522 1.530262 .079701 2.000498 .079701C2.375093 .079701 2.646077-.087671 2.83736-.270984C2.725778 .175342 2.646077 .486177 2.343213 .868742C2.080199 1.195517 1.761395 1.40274 1.40274 1.40274C1.267248 1.40274 .964384 1.378829 .804981 1.139726C1.227397 1.107846 1.259278 .749191 1.259278 .70137C1.259278 .510087 1.115816 .406476 .948443 .406476C.773101 .406476 .494147 .541968 .494147 .932503C.494147 1.307098 .836862 1.625903 1.40274 1.625903C2.215691 1.625903 3.132254 .972354 3.371357 .00797L4.128518-3.004732Z'/> +<path id='g7-40' d='M3.88543 2.905106C3.88543 2.86924 3.88543 2.84533 3.682192 2.642092C2.486675 1.43462 1.817186-.537983 1.817186-2.976837C1.817186-5.296139 2.379078-7.292653 3.765878-8.703362C3.88543-8.810959 3.88543-8.834869 3.88543-8.870735C3.88543-8.942466 3.825654-8.966376 3.777833-8.966376C3.622416-8.966376 2.642092-8.105604 2.056289-6.933998C1.446575-5.726526 1.171606-4.447323 1.171606-2.976837C1.171606-1.912827 1.338979-.490162 1.960648 .789041C2.666002 2.223661 3.646326 3.000747 3.777833 3.000747C3.825654 3.000747 3.88543 2.976837 3.88543 2.905106Z'/> +<path id='g7-41' d='M3.371357-2.976837C3.371357-3.88543 3.251806-5.36787 2.582316-6.75467C1.876961-8.18929 .896638-8.966376 .765131-8.966376C.71731-8.966376 .657534-8.942466 .657534-8.870735C.657534-8.834869 .657534-8.810959 .860772-8.607721C2.056289-7.400249 2.725778-5.427646 2.725778-2.988792C2.725778-.669489 2.163885 1.327024 .777086 2.737733C.657534 2.84533 .657534 2.86924 .657534 2.905106C.657534 2.976837 .71731 3.000747 .765131 3.000747C.920548 3.000747 1.900872 2.139975 2.486675 .968369C3.096389-.251059 3.371357-1.542217 3.371357-2.976837Z'/> +<path id='g7-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +<path id='g7-97' d='M4.614695-3.19203C4.614695-3.837609 4.614695-4.315816 4.088667-4.782067C3.670237-5.164633 3.132254-5.332005 2.606227-5.332005C1.625903-5.332005 .872727-4.686426 .872727-3.90934C.872727-3.56264 1.099875-3.395268 1.374844-3.395268C1.661768-3.395268 1.865006-3.598506 1.865006-3.88543C1.865006-4.375592 1.43462-4.375592 1.255293-4.375592C1.530262-4.877709 2.10411-5.092902 2.582316-5.092902C3.132254-5.092902 3.837609-4.638605 3.837609-3.56264V-3.084433C1.43462-3.048568 .526027-2.044334 .526027-1.123786C.526027-.179328 1.625903 .119552 2.355168 .119552C3.144209 .119552 3.682192-.358655 3.90934-.932503C3.957161-.37061 4.327771 .059776 4.841843 .059776C5.092902 .059776 5.786301-.107597 5.786301-1.06401V-1.733499H5.523288V-1.06401C5.523288-.382565 5.236364-.286924 5.068991-.286924C4.614695-.286924 4.614695-.920548 4.614695-1.099875V-3.19203ZM3.837609-1.685679C3.837609-.514072 2.964882-.119552 2.450809-.119552C1.865006-.119552 1.374844-.549938 1.374844-1.123786C1.374844-2.701868 3.407223-2.84533 3.837609-2.86924V-1.685679Z'/> +<path id='g7-103' d='M1.422665-2.163885C1.984558-1.793275 2.462765-1.793275 2.594271-1.793275C3.670237-1.793275 4.471233-2.606227 4.471233-3.526775C4.471233-3.849564 4.375592-4.303861 3.993026-4.686426C4.459278-5.164633 5.021171-5.164633 5.080946-5.164633C5.128767-5.164633 5.188543-5.164633 5.236364-5.140722C5.116812-5.092902 5.057036-4.97335 5.057036-4.841843C5.057036-4.674471 5.176588-4.531009 5.36787-4.531009C5.463512-4.531009 5.678705-4.590785 5.678705-4.853798C5.678705-5.068991 5.511333-5.403736 5.092902-5.403736C4.471233-5.403736 4.004981-5.021171 3.837609-4.841843C3.478954-5.116812 3.060523-5.272229 2.606227-5.272229C1.530262-5.272229 .729265-4.459278 .729265-3.53873C.729265-2.857285 1.147696-2.414944 1.267248-2.307347C1.123786-2.12802 .908593-1.78132 .908593-1.315068C.908593-.621669 1.327024-.32279 1.422665-.263014C.872727-.107597 .32279 .32279 .32279 .944458C.32279 1.769365 1.446575 2.450809 2.917061 2.450809C4.339726 2.450809 5.523288 1.817186 5.523288 .920548C5.523288 .621669 5.439601-.083686 4.722291-.454296C4.112578-.765131 3.514819-.765131 2.486675-.765131C1.75741-.765131 1.673724-.765131 1.458531-.992279C1.338979-1.111831 1.231382-1.338979 1.231382-1.590037C1.231382-1.793275 1.303113-1.996513 1.422665-2.163885ZM2.606227-2.044334C1.554172-2.044334 1.554172-3.251806 1.554172-3.526775C1.554172-3.741968 1.554172-4.23213 1.75741-4.554919C1.984558-4.901619 2.343213-5.021171 2.594271-5.021171C3.646326-5.021171 3.646326-3.813699 3.646326-3.53873C3.646326-3.323537 3.646326-2.833375 3.443088-2.510585C3.21594-2.163885 2.857285-2.044334 2.606227-2.044334ZM2.929016 2.199751C1.78132 2.199751 .908593 1.613948 .908593 .932503C.908593 .836862 .932503 .37061 1.3868 .059776C1.649813-.107597 1.75741-.107597 2.594271-.107597C3.58655-.107597 4.937484-.107597 4.937484 .932503C4.937484 1.637858 4.028892 2.199751 2.929016 2.199751Z'/> +<path id='g7-108' d='M2.056289-8.296887L.394521-8.16538V-7.81868C1.207472-7.81868 1.303113-7.734994 1.303113-7.149191V-.884682C1.303113-.3467 1.171606-.3467 .394521-.3467V0C.729265-.02391 1.315068-.02391 1.673724-.02391S2.630137-.02391 2.964882 0V-.3467C2.199751-.3467 2.056289-.3467 2.056289-.884682V-8.296887Z'/> +<path id='g7-109' d='M8.571856-2.905106C8.571856-4.016936 8.571856-4.351681 8.296887-4.734247C7.950187-5.200498 7.388294-5.272229 6.981818-5.272229C5.989539-5.272229 5.487422-4.554919 5.296139-4.088667C5.128767-5.009215 4.483188-5.272229 3.730012-5.272229C2.570361-5.272229 2.116065-4.27995 2.020423-4.040847H2.008468V-5.272229L.382565-5.140722V-4.794022C1.195517-4.794022 1.291158-4.710336 1.291158-4.124533V-.884682C1.291158-.3467 1.159651-.3467 .382565-.3467V0C.6934-.02391 1.338979-.02391 1.673724-.02391C2.020423-.02391 2.666002-.02391 2.976837 0V-.3467C2.211706-.3467 2.068244-.3467 2.068244-.884682V-3.108344C2.068244-4.363636 2.893151-5.033126 3.634371-5.033126S4.542964-4.423412 4.542964-3.694147V-.884682C4.542964-.3467 4.411457-.3467 3.634371-.3467V0C3.945205-.02391 4.590785-.02391 4.925529-.02391C5.272229-.02391 5.917808-.02391 6.228643 0V-.3467C5.463512-.3467 5.32005-.3467 5.32005-.884682V-3.108344C5.32005-4.363636 6.144956-5.033126 6.886177-5.033126S7.79477-4.423412 7.79477-3.694147V-.884682C7.79477-.3467 7.663263-.3467 6.886177-.3467V0C7.197011-.02391 7.84259-.02391 8.177335-.02391C8.524035-.02391 9.169614-.02391 9.480448 0V-.3467C8.88269-.3467 8.583811-.3467 8.571856-.705355V-2.905106Z'/> +<path id='g7-111' d='M5.487422-2.558406C5.487422-4.100623 4.315816-5.332005 2.929016-5.332005C1.494396-5.332005 .358655-4.064757 .358655-2.558406C.358655-1.028144 1.554172 .119552 2.917061 .119552C4.327771 .119552 5.487422-1.052055 5.487422-2.558406ZM2.929016-.143462C2.486675-.143462 1.948692-.334745 1.601993-.920548C1.279203-1.458531 1.267248-2.163885 1.267248-2.666002C1.267248-3.120299 1.267248-3.849564 1.637858-4.387547C1.972603-4.901619 2.49863-5.092902 2.917061-5.092902C3.383313-5.092902 3.88543-4.877709 4.208219-4.411457C4.578829-3.861519 4.578829-3.108344 4.578829-2.666002C4.578829-2.247572 4.578829-1.506351 4.267995-.944458C3.93325-.37061 3.383313-.143462 2.929016-.143462Z'/> +<path id='g7-114' d='M1.996513-2.785554C1.996513-3.945205 2.47472-5.033126 3.395268-5.033126C3.490909-5.033126 3.514819-5.033126 3.56264-5.021171C3.466999-4.97335 3.275716-4.901619 3.275716-4.578829C3.275716-4.23213 3.550685-4.100623 3.741968-4.100623C3.981071-4.100623 4.220174-4.25604 4.220174-4.578829C4.220174-4.937484 3.897385-5.272229 3.383313-5.272229C2.367123-5.272229 2.020423-4.172354 1.948692-3.945205H1.936737V-5.272229L.334745-5.140722V-4.794022C1.147696-4.794022 1.243337-4.710336 1.243337-4.124533V-.884682C1.243337-.3467 1.111831-.3467 .334745-.3467V0C.669489-.02391 1.327024-.02391 1.685679-.02391C2.008468-.02391 2.857285-.02391 3.132254 0V-.3467H2.893151C2.020423-.3467 1.996513-.478207 1.996513-.908593V-2.785554Z'/> +<path id='g7-120' d='M3.347447-2.82142C3.694147-3.275716 4.196264-3.921295 4.423412-4.172354C4.913574-4.722291 5.475467-4.805978 5.858032-4.805978V-5.152677C5.34396-5.128767 5.32005-5.128767 4.853798-5.128767C4.399502-5.128767 4.375592-5.128767 3.777833-5.152677V-4.805978C3.93325-4.782067 4.124533-4.710336 4.124533-4.435367C4.124533-4.23213 4.016936-4.100623 3.945205-4.004981L3.180075-3.036613L2.247572-4.267995C2.211706-4.315816 2.139975-4.423412 2.139975-4.507098C2.139975-4.578829 2.199751-4.794022 2.558406-4.805978V-5.152677C2.259527-5.128767 1.649813-5.128767 1.327024-5.128767C.932503-5.128767 .908593-5.128767 .179328-5.152677V-4.805978C.789041-4.805978 1.016189-4.782067 1.267248-4.459278L2.666002-2.630137C2.689913-2.606227 2.737733-2.534496 2.737733-2.49863S1.80523-1.291158 1.685679-1.135741C1.159651-.490162 .633624-.358655 .119552-.3467V0C.573848-.02391 .597758-.02391 1.111831-.02391C1.566127-.02391 1.590037-.02391 2.187796 0V-.3467C1.900872-.382565 1.853051-.561893 1.853051-.729265C1.853051-.920548 1.936737-1.016189 2.056289-1.171606C2.235616-1.422665 2.630137-1.912827 2.917061-2.283437L3.897385-1.004234C4.100623-.74122 4.100623-.71731 4.100623-.645579C4.100623-.549938 4.004981-.358655 3.682192-.3467V0C3.993026-.02391 4.578829-.02391 4.913574-.02391C5.308095-.02391 5.332005-.02391 6.049315 0V-.3467C5.415691-.3467 5.200498-.37061 4.913574-.753176L3.347447-2.82142Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.442282)'> +<use x='56.413267' y='76.479924' xlink:href='#g7-97'/> +<use x='62.266257' y='76.479924' xlink:href='#g7-114'/> +<use x='66.818583' y='76.479924' xlink:href='#g7-103'/> +<use x='74.826654' y='76.479924' xlink:href='#g7-109'/> +<use x='84.581638' y='76.479924' xlink:href='#g7-97'/> +<use x='90.434628' y='76.479924' xlink:href='#g7-120'/> +<use x='83.483872' y='83.653022' xlink:href='#g4-121'/> +<use x='102.093757' y='68.392165' xlink:href='#g7-49'/> +<rect x='97.808298' y='73.252038' height='.478187' width='14.423907'/> +<use x='97.808298' y='84.680586' xlink:href='#g5-84'/> +<use x='106.295134' y='81.226878' xlink:href='#g4-11'/> +<use x='104.669303' y='87.636102' xlink:href='#g4-121'/> +<use x='119.340924' y='60.440006' xlink:href='#g4-84'/> +<use x='124.27112' y='61.43627' xlink:href='#g3-121'/> +<use x='115.420217' y='65.122459' xlink:href='#g0-88'/> +<use x='117.11517' y='90.228446' xlink:href='#g4-116'/> +<use x='120.173191' y='90.228446' xlink:href='#g6-61'/> +<use x='126.759698' y='90.228446' xlink:href='#g6-49'/> +<use x='134.681331' y='76.479924' xlink:href='#g7-108'/> +<use x='137.932992' y='76.479924' xlink:href='#g7-111'/> +<use x='143.785983' y='76.479924' xlink:href='#g7-103'/> +<use x='153.122385' y='76.479924' xlink:href='#g5-112'/> +<use x='158.997528' y='76.479924' xlink:href='#g7-40'/> +<use x='163.549854' y='76.479924' xlink:href='#g5-121'/> +<use x='169.686506' y='71.543738' xlink:href='#g4-60'/> +<use x='176.273012' y='71.543738' xlink:href='#g4-116'/> +<use x='179.331033' y='71.543738' xlink:href='#g4-62'/> +<use x='186.415672' y='76.479924' xlink:href='#g2-106'/> +<use x='189.736562' y='76.479924' xlink:href='#g5-120'/> +<use x='196.388649' y='76.479924' xlink:href='#g5-59'/> +<use x='201.632808' y='76.479924' xlink:href='#g5-121'/> +<use x='207.76946' y='71.543738' xlink:href='#g4-60'/> +<use x='214.355967' y='71.543738' xlink:href='#g6-49'/> +<use x='218.590149' y='71.543738' xlink:href='#g4-62'/> +<use x='225.674788' y='76.479924' xlink:href='#g5-59'/> +<use x='230.918947' y='76.479924' xlink:href='#g5-58'/> +<use x='234.170608' y='76.479924' xlink:href='#g5-58'/> +<use x='237.422269' y='76.479924' xlink:href='#g5-58'/> +<use x='240.673931' y='76.479924' xlink:href='#g5-59'/> +<use x='245.91809' y='76.479924' xlink:href='#g5-121'/> +<use x='252.054741' y='71.543738' xlink:href='#g4-60'/> +<use x='258.641248' y='71.543738' xlink:href='#g4-116'/> +<use x='261.699269' y='71.543738' xlink:href='#g1-0'/> +<use x='268.285776' y='71.543738' xlink:href='#g6-49'/> +<use x='272.519958' y='71.543738' xlink:href='#g4-62'/> +<use x='279.604597' y='76.479924' xlink:href='#g7-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/beam-search-p.svg b/_resources/beam-search-p.svg new file mode 100644 index 0000000..641bde0 --- /dev/null +++ b/_resources/beam-search-p.svg @@ -0,0 +1,97 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='392.8276pt' height='39.839208pt' viewBox='-.239051 -.233284 392.8276 39.839208'> +<defs> +<path id='g1-0' d='M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z'/> +<path id='g2-106' d='M1.900872-8.53599C1.900872-8.751183 1.900872-8.966376 1.661768-8.966376S1.422665-8.751183 1.422665-8.53599V2.558406C1.422665 2.773599 1.422665 2.988792 1.661768 2.988792S1.900872 2.773599 1.900872 2.558406V-8.53599Z'/> +<path id='g0-89' d='M14.59726 16.737235V16.091656C13.007223 16.091656 12.636613 15.541719 12.636613 14.776588V1.960648C12.636613 1.183562 13.019178 .645579 14.59726 .645579V0H.669489V.645579C2.259527 .645579 2.630137 1.195517 2.630137 1.960648V14.776588C2.630137 15.553674 2.247572 16.091656 .669489 16.091656V16.737235H6.419925V16.091656C4.829888 16.091656 4.459278 15.541719 4.459278 14.776588V.645579H10.807472V14.776588C10.807472 15.553674 10.424907 16.091656 8.846824 16.091656V16.737235H14.59726Z'/> +<path id='g3-121' d='M3.56264-2.295392C3.574595-2.337235 3.580573-2.361146 3.580573-2.379078C3.580573-2.450809 3.526775-2.576339 3.36538-2.576339C3.263761-2.576339 3.162142-2.510585 3.120299-2.426899C3.096389-2.385056 3.054545-2.199751 3.024658-2.086177L2.899128-1.590037L2.713823-.842839C2.67198-.675467 2.64807-.579826 2.408966-.352677C2.343213-.292902 2.15193-.137484 1.894894-.137484C1.452553-.137484 1.452553-.526027 1.452553-.633624C1.452553-.896638 1.524284-1.165629 1.787298-1.823163C1.847073-1.966625 1.865006-2.008468 1.865006-2.116065C1.865006-2.450809 1.566127-2.636115 1.267248-2.636115C.657534-2.636115 .364633-1.847073 .364633-1.715567C.364633-1.685679 .388543-1.63188 .472229-1.63188S.573848-1.667746 .591781-1.721544C.759153-2.30137 1.075965-2.438854 1.243337-2.438854C1.362889-2.438854 1.404732-2.361146 1.404732-2.223661C1.404732-2.098132 1.327024-1.894894 1.26127-1.733499C1.052055-1.207472 .980324-.932503 .980324-.71731C.980324-.14944 1.404732 .059776 1.870984 .059776C1.95467 .059776 2.235616 .059776 2.552428-.215193C2.47472 .113574 2.420922 .3467 2.133998 .657534C2.026401 .777086 1.75741 1.022167 1.398755 1.022167C1.344956 1.022167 1.08792 1.016189 .950436 .860772C1.21345 .812951 1.249315 .579826 1.249315 .526027C1.249315 .364633 1.117808 .298879 1.004234 .298879C.854795 .298879 .657534 .418431 .657534 .6934C.657534 .980324 .938481 1.219427 1.404732 1.219427C2.038356 1.219427 2.803487 .753176 2.988792 .005978L3.56264-2.295392Z'/> +<path id='g6-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g6-61' d='M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z'/> +<path id='g5-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/> +<path id='g5-59' d='M2.331258 .047821C2.331258-.645579 2.10411-1.159651 1.613948-1.159651C1.231382-1.159651 1.0401-.848817 1.0401-.585803S1.219427 0 1.625903 0C1.78132 0 1.912827-.047821 2.020423-.155417C2.044334-.179328 2.056289-.179328 2.068244-.179328C2.092154-.179328 2.092154-.011955 2.092154 .047821C2.092154 .442341 2.020423 1.219427 1.327024 1.996513C1.195517 2.139975 1.195517 2.163885 1.195517 2.187796C1.195517 2.247572 1.255293 2.307347 1.315068 2.307347C1.41071 2.307347 2.331258 1.422665 2.331258 .047821Z'/> +<path id='g5-112' d='M.514072 1.518306C.430386 1.876961 .382565 1.972603-.107597 1.972603C-.251059 1.972603-.37061 1.972603-.37061 2.199751C-.37061 2.223661-.358655 2.319303-.227148 2.319303C-.071731 2.319303 .095641 2.295392 .251059 2.295392H.765131C1.016189 2.295392 1.625903 2.319303 1.876961 2.319303C1.948692 2.319303 2.092154 2.319303 2.092154 2.10411C2.092154 1.972603 2.008468 1.972603 1.80523 1.972603C1.255293 1.972603 1.219427 1.888917 1.219427 1.793275C1.219427 1.649813 1.75741-.406476 1.829141-.681445C1.960648-.3467 2.283437 .119552 2.905106 .119552C4.25604 .119552 5.71457-1.637858 5.71457-3.395268C5.71457-4.495143 5.092902-5.272229 4.196264-5.272229C3.431133-5.272229 2.785554-4.531009 2.654047-4.363636C2.558406-4.961395 2.092154-5.272229 1.613948-5.272229C1.267248-5.272229 .992279-5.104857 .765131-4.65056C.549938-4.220174 .382565-3.490909 .382565-3.443088S.430386-3.335492 .514072-3.335492C.609714-3.335492 .621669-3.347447 .6934-3.622416C.872727-4.327771 1.099875-5.033126 1.578082-5.033126C1.853051-5.033126 1.948692-4.841843 1.948692-4.483188C1.948692-4.196264 1.912827-4.076712 1.865006-3.861519L.514072 1.518306ZM2.582316-3.730012C2.666002-4.064757 3.000747-4.411457 3.19203-4.578829C3.323537-4.698381 3.718057-5.033126 4.172354-5.033126C4.698381-5.033126 4.937484-4.507098 4.937484-3.88543C4.937484-3.311582 4.60274-1.960648 4.303861-1.338979C4.004981-.6934 3.455044-.119552 2.905106-.119552C2.092154-.119552 1.960648-1.147696 1.960648-1.195517C1.960648-1.231382 1.984558-1.327024 1.996513-1.3868L2.582316-3.730012Z'/> +<path id='g5-120' d='M5.66675-4.877709C5.284184-4.805978 5.140722-4.519054 5.140722-4.291905C5.140722-4.004981 5.36787-3.90934 5.535243-3.90934C5.893898-3.90934 6.144956-4.220174 6.144956-4.542964C6.144956-5.045081 5.571108-5.272229 5.068991-5.272229C4.339726-5.272229 3.93325-4.554919 3.825654-4.327771C3.550685-5.224408 2.809465-5.272229 2.594271-5.272229C1.374844-5.272229 .729265-3.706102 .729265-3.443088C.729265-3.395268 .777086-3.335492 .860772-3.335492C.956413-3.335492 .980324-3.407223 1.004234-3.455044C1.41071-4.782067 2.211706-5.033126 2.558406-5.033126C3.096389-5.033126 3.203985-4.531009 3.203985-4.244085C3.203985-3.981071 3.132254-3.706102 2.988792-3.132254L2.582316-1.494396C2.402989-.777086 2.056289-.119552 1.422665-.119552C1.362889-.119552 1.06401-.119552 .812951-.274969C1.243337-.358655 1.338979-.71731 1.338979-.860772C1.338979-1.099875 1.159651-1.243337 .932503-1.243337C.645579-1.243337 .334745-.992279 .334745-.609714C.334745-.107597 .896638 .119552 1.41071 .119552C1.984558 .119552 2.391034-.334745 2.642092-.824907C2.833375-.119552 3.431133 .119552 3.873474 .119552C5.092902 .119552 5.738481-1.446575 5.738481-1.709589C5.738481-1.769365 5.69066-1.817186 5.618929-1.817186C5.511333-1.817186 5.499377-1.75741 5.463512-1.661768C5.140722-.609714 4.447323-.119552 3.90934-.119552C3.490909-.119552 3.263761-.430386 3.263761-.920548C3.263761-1.183562 3.311582-1.374844 3.502864-2.163885L3.921295-3.789788C4.100623-4.507098 4.507098-5.033126 5.057036-5.033126C5.080946-5.033126 5.415691-5.033126 5.66675-4.877709Z'/> +<path id='g5-121' d='M3.144209 1.338979C2.82142 1.793275 2.355168 2.199751 1.769365 2.199751C1.625903 2.199751 1.052055 2.175841 .872727 1.625903C.908593 1.637858 .968369 1.637858 .992279 1.637858C1.350934 1.637858 1.590037 1.327024 1.590037 1.052055S1.362889 .681445 1.183562 .681445C.992279 .681445 .573848 .824907 .573848 1.41071C.573848 2.020423 1.08792 2.438854 1.769365 2.438854C2.964882 2.438854 4.172354 1.338979 4.507098 .011955L5.678705-4.65056C5.69066-4.710336 5.71457-4.782067 5.71457-4.853798C5.71457-5.033126 5.571108-5.152677 5.391781-5.152677C5.284184-5.152677 5.033126-5.104857 4.937484-4.746202L4.052802-1.231382C3.993026-1.016189 3.993026-.992279 3.897385-.860772C3.658281-.526027 3.263761-.119552 2.689913-.119552C2.020423-.119552 1.960648-.777086 1.960648-1.099875C1.960648-1.78132 2.283437-2.701868 2.606227-3.56264C2.737733-3.90934 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.033126 2.139975-4.638605C2.139975-4.327771 2.008468-3.981071 1.829141-3.526775C1.243337-1.960648 1.243337-1.566127 1.243337-1.279203C1.243337-.143462 2.056289 .119552 2.654047 .119552C3.000747 .119552 3.431133 .011955 3.849564-.430386L3.861519-.418431C3.682192 .286924 3.56264 .753176 3.144209 1.338979Z'/> +<path id='g4-60' d='M5.7066-4.112578C5.802242-4.160399 5.873973-4.208219 5.873973-4.311831S5.794271-4.495143 5.69066-4.495143C5.66675-4.495143 5.650809-4.495143 5.547198-4.439352L.868742-2.191781C.773101-2.14396 .70137-2.096139 .70137-1.992528S.773101-1.841096 .868742-1.793275L5.547198 .454296C5.650809 .510087 5.66675 .510087 5.69066 .510087C5.794271 .510087 5.873973 .430386 5.873973 .326775S5.802242 .175342 5.7066 .127522L1.307098-1.992528L5.7066-4.112578Z'/> +<path id='g4-62' d='M5.7066-1.793275C5.802242-1.841096 5.873973-1.888917 5.873973-1.992528S5.802242-2.14396 5.7066-2.191781L1.028144-4.439352C.924533-4.495143 .908593-4.495143 .884682-4.495143C.781071-4.495143 .70137-4.415442 .70137-4.311831S.773101-4.160399 .868742-4.112578L5.268244-1.992528L.868742 .127522C.773101 .175342 .70137 .223163 .70137 .326775S.781071 .510087 .884682 .510087C.908593 .510087 .924533 .510087 1.028144 .454296L5.7066-1.793275Z'/> +<path id='g4-84' d='M3.602491-4.821918C3.674222-5.108842 3.682192-5.124782 4.008966-5.124782H4.614695C5.443587-5.124782 5.539228-4.861768 5.539228-4.463263C5.539228-4.26401 5.491407-3.921295 5.483437-3.881445C5.467497-3.793773 5.459527-3.722042 5.459527-3.706102C5.459527-3.602491 5.531258-3.57858 5.579078-3.57858C5.66675-3.57858 5.69863-3.626401 5.72254-3.777833L5.937733-5.276214C5.937733-5.387796 5.842092-5.387796 5.69863-5.387796H1.004234C.804981-5.387796 .789041-5.387796 .73325-5.220423L.247073-3.841594C.231133-3.801743 .207223-3.737983 .207223-3.690162C.207223-3.626401 .263014-3.57858 .326775-3.57858C.414446-3.57858 .430386-3.618431 .478207-3.753923C.932503-5.029141 1.163636-5.124782 2.375093-5.124782H2.685928C2.925031-5.124782 2.933001-5.116812 2.933001-5.053051C2.933001-5.029141 2.901121-4.869738 2.893151-4.837858L1.841096-.653549C1.769365-.350685 1.745455-.263014 .916563-.263014C.661519-.263014 .581818-.263014 .581818-.111582C.581818-.103611 .581818 0 .71731 0C.932503 0 1.482441-.02391 1.697634-.02391H2.375093C2.598257-.02391 3.156164 0 3.379328 0C3.443088 0 3.56264 0 3.56264-.151432C3.56264-.263014 3.474969-.263014 3.259776-.263014C3.068493-.263014 3.004732-.263014 2.797509-.278954C2.542466-.302864 2.510585-.334745 2.510585-.438356C2.510585-.470237 2.518555-.502117 2.542466-.581818L3.602491-4.821918Z'/> +<path id='g4-116' d='M1.761395-3.172105H2.542466C2.693898-3.172105 2.789539-3.172105 2.789539-3.323537C2.789539-3.435118 2.685928-3.435118 2.550436-3.435118H1.825156L2.11208-4.566874C2.14396-4.686426 2.14396-4.726276 2.14396-4.734247C2.14396-4.901619 2.016438-4.98132 1.880946-4.98132C1.609963-4.98132 1.554172-4.766127 1.466501-4.407472L1.219427-3.435118H.454296C.302864-3.435118 .199253-3.435118 .199253-3.283686C.199253-3.172105 .302864-3.172105 .438356-3.172105H1.155666L.67746-1.259278C.629639-1.060025 .557908-.781071 .557908-.669489C.557908-.191283 .948443 .079701 1.370859 .079701C2.223661 .079701 2.709838-1.044085 2.709838-1.139726C2.709838-1.227397 2.638107-1.243337 2.590286-1.243337C2.502615-1.243337 2.494645-1.211457 2.438854-1.091905C2.279452-.70934 1.880946-.143462 1.39477-.143462C1.227397-.143462 1.131756-.255044 1.131756-.518057C1.131756-.669489 1.155666-.757161 1.179577-.860772L1.761395-3.172105Z'/> +<path id='g4-121' d='M4.128518-3.004732C4.160399-3.116314 4.160399-3.132254 4.160399-3.188045C4.160399-3.387298 4.000996-3.435118 3.905355-3.435118C3.865504-3.435118 3.682192-3.427148 3.57858-3.219925C3.56264-3.180075 3.490909-2.893151 3.451059-2.725778L2.972852-.812951C2.964882-.789041 2.622167-.143462 2.040349-.143462C1.649813-.143462 1.514321-.430386 1.514321-.789041C1.514321-1.251308 1.785305-1.960648 1.968618-2.422914C2.048319-2.622167 2.072229-2.693898 2.072229-2.83736C2.072229-3.275716 1.721544-3.514819 1.354919-3.514819C.565878-3.514819 .239103-2.391034 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.70137-3.012702 1.044085-3.291656 1.331009-3.291656C1.45056-3.291656 1.522291-3.211955 1.522291-3.028643C1.522291-2.86127 1.458531-2.677958 1.40274-2.534496C1.075965-1.689664 .940473-1.283188 .940473-.908593C.940473-.127522 1.530262 .079701 2.000498 .079701C2.375093 .079701 2.646077-.087671 2.83736-.270984C2.725778 .175342 2.646077 .486177 2.343213 .868742C2.080199 1.195517 1.761395 1.40274 1.40274 1.40274C1.267248 1.40274 .964384 1.378829 .804981 1.139726C1.227397 1.107846 1.259278 .749191 1.259278 .70137C1.259278 .510087 1.115816 .406476 .948443 .406476C.773101 .406476 .494147 .541968 .494147 .932503C.494147 1.307098 .836862 1.625903 1.40274 1.625903C2.215691 1.625903 3.132254 .972354 3.371357 .00797L4.128518-3.004732Z'/> +<path id='g7-40' d='M3.88543 2.905106C3.88543 2.86924 3.88543 2.84533 3.682192 2.642092C2.486675 1.43462 1.817186-.537983 1.817186-2.976837C1.817186-5.296139 2.379078-7.292653 3.765878-8.703362C3.88543-8.810959 3.88543-8.834869 3.88543-8.870735C3.88543-8.942466 3.825654-8.966376 3.777833-8.966376C3.622416-8.966376 2.642092-8.105604 2.056289-6.933998C1.446575-5.726526 1.171606-4.447323 1.171606-2.976837C1.171606-1.912827 1.338979-.490162 1.960648 .789041C2.666002 2.223661 3.646326 3.000747 3.777833 3.000747C3.825654 3.000747 3.88543 2.976837 3.88543 2.905106Z'/> +<path id='g7-41' d='M3.371357-2.976837C3.371357-3.88543 3.251806-5.36787 2.582316-6.75467C1.876961-8.18929 .896638-8.966376 .765131-8.966376C.71731-8.966376 .657534-8.942466 .657534-8.870735C.657534-8.834869 .657534-8.810959 .860772-8.607721C2.056289-7.400249 2.725778-5.427646 2.725778-2.988792C2.725778-.669489 2.163885 1.327024 .777086 2.737733C.657534 2.84533 .657534 2.86924 .657534 2.905106C.657534 2.976837 .71731 3.000747 .765131 3.000747C.920548 3.000747 1.900872 2.139975 2.486675 .968369C3.096389-.251059 3.371357-1.542217 3.371357-2.976837Z'/> +<path id='g7-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g7-97' d='M4.614695-3.19203C4.614695-3.837609 4.614695-4.315816 4.088667-4.782067C3.670237-5.164633 3.132254-5.332005 2.606227-5.332005C1.625903-5.332005 .872727-4.686426 .872727-3.90934C.872727-3.56264 1.099875-3.395268 1.374844-3.395268C1.661768-3.395268 1.865006-3.598506 1.865006-3.88543C1.865006-4.375592 1.43462-4.375592 1.255293-4.375592C1.530262-4.877709 2.10411-5.092902 2.582316-5.092902C3.132254-5.092902 3.837609-4.638605 3.837609-3.56264V-3.084433C1.43462-3.048568 .526027-2.044334 .526027-1.123786C.526027-.179328 1.625903 .119552 2.355168 .119552C3.144209 .119552 3.682192-.358655 3.90934-.932503C3.957161-.37061 4.327771 .059776 4.841843 .059776C5.092902 .059776 5.786301-.107597 5.786301-1.06401V-1.733499H5.523288V-1.06401C5.523288-.382565 5.236364-.286924 5.068991-.286924C4.614695-.286924 4.614695-.920548 4.614695-1.099875V-3.19203ZM3.837609-1.685679C3.837609-.514072 2.964882-.119552 2.450809-.119552C1.865006-.119552 1.374844-.549938 1.374844-1.123786C1.374844-2.701868 3.407223-2.84533 3.837609-2.86924V-1.685679Z'/> +<path id='g7-103' d='M1.422665-2.163885C1.984558-1.793275 2.462765-1.793275 2.594271-1.793275C3.670237-1.793275 4.471233-2.606227 4.471233-3.526775C4.471233-3.849564 4.375592-4.303861 3.993026-4.686426C4.459278-5.164633 5.021171-5.164633 5.080946-5.164633C5.128767-5.164633 5.188543-5.164633 5.236364-5.140722C5.116812-5.092902 5.057036-4.97335 5.057036-4.841843C5.057036-4.674471 5.176588-4.531009 5.36787-4.531009C5.463512-4.531009 5.678705-4.590785 5.678705-4.853798C5.678705-5.068991 5.511333-5.403736 5.092902-5.403736C4.471233-5.403736 4.004981-5.021171 3.837609-4.841843C3.478954-5.116812 3.060523-5.272229 2.606227-5.272229C1.530262-5.272229 .729265-4.459278 .729265-3.53873C.729265-2.857285 1.147696-2.414944 1.267248-2.307347C1.123786-2.12802 .908593-1.78132 .908593-1.315068C.908593-.621669 1.327024-.32279 1.422665-.263014C.872727-.107597 .32279 .32279 .32279 .944458C.32279 1.769365 1.446575 2.450809 2.917061 2.450809C4.339726 2.450809 5.523288 1.817186 5.523288 .920548C5.523288 .621669 5.439601-.083686 4.722291-.454296C4.112578-.765131 3.514819-.765131 2.486675-.765131C1.75741-.765131 1.673724-.765131 1.458531-.992279C1.338979-1.111831 1.231382-1.338979 1.231382-1.590037C1.231382-1.793275 1.303113-1.996513 1.422665-2.163885ZM2.606227-2.044334C1.554172-2.044334 1.554172-3.251806 1.554172-3.526775C1.554172-3.741968 1.554172-4.23213 1.75741-4.554919C1.984558-4.901619 2.343213-5.021171 2.594271-5.021171C3.646326-5.021171 3.646326-3.813699 3.646326-3.53873C3.646326-3.323537 3.646326-2.833375 3.443088-2.510585C3.21594-2.163885 2.857285-2.044334 2.606227-2.044334ZM2.929016 2.199751C1.78132 2.199751 .908593 1.613948 .908593 .932503C.908593 .836862 .932503 .37061 1.3868 .059776C1.649813-.107597 1.75741-.107597 2.594271-.107597C3.58655-.107597 4.937484-.107597 4.937484 .932503C4.937484 1.637858 4.028892 2.199751 2.929016 2.199751Z'/> +<path id='g7-109' d='M8.571856-2.905106C8.571856-4.016936 8.571856-4.351681 8.296887-4.734247C7.950187-5.200498 7.388294-5.272229 6.981818-5.272229C5.989539-5.272229 5.487422-4.554919 5.296139-4.088667C5.128767-5.009215 4.483188-5.272229 3.730012-5.272229C2.570361-5.272229 2.116065-4.27995 2.020423-4.040847H2.008468V-5.272229L.382565-5.140722V-4.794022C1.195517-4.794022 1.291158-4.710336 1.291158-4.124533V-.884682C1.291158-.3467 1.159651-.3467 .382565-.3467V0C.6934-.02391 1.338979-.02391 1.673724-.02391C2.020423-.02391 2.666002-.02391 2.976837 0V-.3467C2.211706-.3467 2.068244-.3467 2.068244-.884682V-3.108344C2.068244-4.363636 2.893151-5.033126 3.634371-5.033126S4.542964-4.423412 4.542964-3.694147V-.884682C4.542964-.3467 4.411457-.3467 3.634371-.3467V0C3.945205-.02391 4.590785-.02391 4.925529-.02391C5.272229-.02391 5.917808-.02391 6.228643 0V-.3467C5.463512-.3467 5.32005-.3467 5.32005-.884682V-3.108344C5.32005-4.363636 6.144956-5.033126 6.886177-5.033126S7.79477-4.423412 7.79477-3.694147V-.884682C7.79477-.3467 7.663263-.3467 6.886177-.3467V0C7.197011-.02391 7.84259-.02391 8.177335-.02391C8.524035-.02391 9.169614-.02391 9.480448 0V-.3467C8.88269-.3467 8.583811-.3467 8.571856-.705355V-2.905106Z'/> +<path id='g7-114' d='M1.996513-2.785554C1.996513-3.945205 2.47472-5.033126 3.395268-5.033126C3.490909-5.033126 3.514819-5.033126 3.56264-5.021171C3.466999-4.97335 3.275716-4.901619 3.275716-4.578829C3.275716-4.23213 3.550685-4.100623 3.741968-4.100623C3.981071-4.100623 4.220174-4.25604 4.220174-4.578829C4.220174-4.937484 3.897385-5.272229 3.383313-5.272229C2.367123-5.272229 2.020423-4.172354 1.948692-3.945205H1.936737V-5.272229L.334745-5.140722V-4.794022C1.147696-4.794022 1.243337-4.710336 1.243337-4.124533V-.884682C1.243337-.3467 1.111831-.3467 .334745-.3467V0C.669489-.02391 1.327024-.02391 1.685679-.02391C2.008468-.02391 2.857285-.02391 3.132254 0V-.3467H2.893151C2.020423-.3467 1.996513-.478207 1.996513-.908593V-2.785554Z'/> +<path id='g7-120' d='M3.347447-2.82142C3.694147-3.275716 4.196264-3.921295 4.423412-4.172354C4.913574-4.722291 5.475467-4.805978 5.858032-4.805978V-5.152677C5.34396-5.128767 5.32005-5.128767 4.853798-5.128767C4.399502-5.128767 4.375592-5.128767 3.777833-5.152677V-4.805978C3.93325-4.782067 4.124533-4.710336 4.124533-4.435367C4.124533-4.23213 4.016936-4.100623 3.945205-4.004981L3.180075-3.036613L2.247572-4.267995C2.211706-4.315816 2.139975-4.423412 2.139975-4.507098C2.139975-4.578829 2.199751-4.794022 2.558406-4.805978V-5.152677C2.259527-5.128767 1.649813-5.128767 1.327024-5.128767C.932503-5.128767 .908593-5.128767 .179328-5.152677V-4.805978C.789041-4.805978 1.016189-4.782067 1.267248-4.459278L2.666002-2.630137C2.689913-2.606227 2.737733-2.534496 2.737733-2.49863S1.80523-1.291158 1.685679-1.135741C1.159651-.490162 .633624-.358655 .119552-.3467V0C.573848-.02391 .597758-.02391 1.111831-.02391C1.566127-.02391 1.590037-.02391 2.187796 0V-.3467C1.900872-.382565 1.853051-.561893 1.853051-.729265C1.853051-.920548 1.936737-1.016189 2.056289-1.171606C2.235616-1.422665 2.630137-1.912827 2.917061-2.283437L3.897385-1.004234C4.100623-.74122 4.100623-.71731 4.100623-.645579C4.100623-.549938 4.004981-.358655 3.682192-.3467V0C3.993026-.02391 4.578829-.02391 4.913574-.02391C5.308095-.02391 5.332005-.02391 6.049315 0V-.3467C5.415691-.3467 5.200498-.37061 4.913574-.753176L3.347447-2.82142Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.442282)'> +<use x='56.413267' y='76.479924' xlink:href='#g7-97'/> +<use x='62.266257' y='76.479924' xlink:href='#g7-114'/> +<use x='66.818583' y='76.479924' xlink:href='#g7-103'/> +<use x='74.826654' y='76.479924' xlink:href='#g7-109'/> +<use x='84.581638' y='76.479924' xlink:href='#g7-97'/> +<use x='90.434628' y='76.479924' xlink:href='#g7-120'/> +<use x='83.483872' y='83.653022' xlink:href='#g4-121'/> +<use x='100.514778' y='76.479924' xlink:href='#g5-112'/> +<use x='106.389921' y='76.479924' xlink:href='#g7-40'/> +<use x='110.942247' y='76.479924' xlink:href='#g5-121'/> +<use x='117.078898' y='71.543738' xlink:href='#g4-60'/> +<use x='123.665405' y='71.543738' xlink:href='#g6-49'/> +<use x='127.899588' y='71.543738' xlink:href='#g4-62'/> +<use x='134.984227' y='76.479924' xlink:href='#g5-59'/> +<use x='140.228385' y='76.479924' xlink:href='#g5-58'/> +<use x='143.480047' y='76.479924' xlink:href='#g5-58'/> +<use x='146.731708' y='76.479924' xlink:href='#g5-58'/> +<use x='149.983369' y='76.479924' xlink:href='#g5-59'/> +<use x='155.227528' y='76.479924' xlink:href='#g5-121'/> +<use x='161.36418' y='71.543738' xlink:href='#g4-60'/> +<use x='167.950686' y='71.543738' xlink:href='#g4-84'/> +<use x='172.880883' y='72.540002' xlink:href='#g3-121'/> +<use x='177.377889' y='71.543738' xlink:href='#g4-62'/> +<use x='184.462528' y='76.479924' xlink:href='#g2-106'/> +<use x='187.783418' y='76.479924' xlink:href='#g5-120'/> +<use x='194.435505' y='76.479924' xlink:href='#g7-41'/> +<use x='202.30866' y='76.479924' xlink:href='#g7-61'/> +<use x='214.734141' y='76.479924' xlink:href='#g7-97'/> +<use x='220.587132' y='76.479924' xlink:href='#g7-114'/> +<use x='225.139457' y='76.479924' xlink:href='#g7-103'/> +<use x='233.147528' y='76.479924' xlink:href='#g7-109'/> +<use x='242.902512' y='76.479924' xlink:href='#g7-97'/> +<use x='248.755502' y='76.479924' xlink:href='#g7-120'/> +<use x='241.804746' y='83.653022' xlink:href='#g4-121'/> +<use x='259.850599' y='60.440006' xlink:href='#g4-84'/> +<use x='264.780796' y='61.43627' xlink:href='#g3-121'/> +<use x='256.926156' y='65.122459' xlink:href='#g0-89'/> +<use x='257.624845' y='90.228446' xlink:href='#g4-116'/> +<use x='260.682866' y='90.228446' xlink:href='#g6-61'/> +<use x='267.269373' y='90.228446' xlink:href='#g6-49'/> +<use x='274.194743' y='76.479924' xlink:href='#g5-112'/> +<use x='280.069886' y='76.479924' xlink:href='#g7-40'/> +<use x='284.622211' y='76.479924' xlink:href='#g5-121'/> +<use x='290.758863' y='71.543738' xlink:href='#g4-60'/> +<use x='297.34537' y='71.543738' xlink:href='#g4-116'/> +<use x='300.403391' y='71.543738' xlink:href='#g4-62'/> +<use x='307.488029' y='76.479924' xlink:href='#g2-106'/> +<use x='310.80892' y='76.479924' xlink:href='#g5-120'/> +<use x='317.461007' y='76.479924' xlink:href='#g5-59'/> +<use x='322.705166' y='76.479924' xlink:href='#g5-121'/> +<use x='328.841817' y='71.543738' xlink:href='#g4-60'/> +<use x='335.428324' y='71.543738' xlink:href='#g6-49'/> +<use x='339.662507' y='71.543738' xlink:href='#g4-62'/> +<use x='346.747145' y='76.479924' xlink:href='#g5-59'/> +<use x='351.991304' y='76.479924' xlink:href='#g5-58'/> +<use x='355.242966' y='76.479924' xlink:href='#g5-58'/> +<use x='358.494627' y='76.479924' xlink:href='#g5-58'/> +<use x='361.746288' y='76.479924' xlink:href='#g5-59'/> +<use x='366.990447' y='76.479924' xlink:href='#g5-121'/> +<use x='373.127099' y='71.543738' xlink:href='#g4-60'/> +<use x='379.713605' y='71.543738' xlink:href='#g4-116'/> +<use x='382.771626' y='71.543738' xlink:href='#g1-0'/> +<use x='389.358133' y='71.543738' xlink:href='#g6-49'/> +<use x='393.592316' y='71.543738' xlink:href='#g4-62'/> +<use x='400.676954' y='76.479924' xlink:href='#g7-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/beb7e37bd0e389910f37d2ef559a1900.png b/_resources/beb7e37bd0e389910f37d2ef559a1900.png new file mode 100644 index 0000000..92e04bb Binary files /dev/null and b/_resources/beb7e37bd0e389910f37d2ef559a1900.png differ diff --git a/_resources/bfd9c881c6664a63e2334228e4e57f83.png b/_resources/bfd9c881c6664a63e2334228e4e57f83.png new file mode 100644 index 0000000..dbdea85 Binary files /dev/null and b/_resources/bfd9c881c6664a63e2334228e4e57f83.png differ diff --git a/_resources/bias-c1.svg b/_resources/bias-c1.svg new file mode 100644 index 0000000..0d39228 --- /dev/null +++ b/_resources/bias-c1.svg @@ -0,0 +1,24 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='64.558507pt' height='11.536963pt' viewBox='-.239051 -.242965 64.558507 11.536963'> +<defs> +<path id='g2-48' d='M5.355915-3.825654C5.355915-4.817933 5.296139-5.786301 4.865753-6.694894C4.375592-7.687173 3.514819-7.950187 2.929016-7.950187C2.235616-7.950187 1.3868-7.603487 .944458-6.611208C.609714-5.858032 .490162-5.116812 .490162-3.825654C.490162-2.666002 .573848-1.793275 1.004234-.944458C1.470486-.035866 2.295392 .251059 2.917061 .251059C3.957161 .251059 4.554919-.37061 4.901619-1.06401C5.332005-1.960648 5.355915-3.132254 5.355915-3.825654ZM2.917061 .011955C2.534496 .011955 1.75741-.203238 1.530262-1.506351C1.398755-2.223661 1.398755-3.132254 1.398755-3.969116C1.398755-4.94944 1.398755-5.834122 1.590037-6.539477C1.793275-7.340473 2.402989-7.711083 2.917061-7.711083C3.371357-7.711083 4.064757-7.436115 4.291905-6.40797C4.447323-5.726526 4.447323-4.782067 4.447323-3.969116C4.447323-3.16812 4.447323-2.259527 4.315816-1.530262C4.088667-.215193 3.335492 .011955 2.917061 .011955Z'/> +<path id='g2-50' d='M5.260274-2.008468H4.99726C4.961395-1.80523 4.865753-1.147696 4.746202-.956413C4.662516-.848817 3.981071-.848817 3.622416-.848817H1.41071C1.733499-1.123786 2.462765-1.888917 2.773599-2.175841C4.590785-3.849564 5.260274-4.471233 5.260274-5.654795C5.260274-7.029639 4.172354-7.950187 2.785554-7.950187S.585803-6.766625 .585803-5.738481C.585803-5.128767 1.111831-5.128767 1.147696-5.128767C1.398755-5.128767 1.709589-5.308095 1.709589-5.69066C1.709589-6.025405 1.482441-6.252553 1.147696-6.252553C1.0401-6.252553 1.016189-6.252553 .980324-6.240598C1.207472-7.053549 1.853051-7.603487 2.630137-7.603487C3.646326-7.603487 4.267995-6.75467 4.267995-5.654795C4.267995-4.638605 3.682192-3.753923 3.000747-2.988792L.585803-.286924V0H4.94944L5.260274-2.008468Z'/> +<path id='g2-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g1-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g0-18' d='M5.296139-6.01345C5.296139-7.232877 4.913574-8.416438 3.93325-8.416438C2.259527-8.416438 .478207-4.913574 .478207-2.283437C.478207-1.733499 .597758 .119552 1.853051 .119552C3.478954 .119552 5.296139-3.299626 5.296139-6.01345ZM1.673724-4.327771C1.853051-5.033126 2.10411-6.03736 2.582316-6.886177C2.976837-7.603487 3.395268-8.177335 3.921295-8.177335C4.315816-8.177335 4.578829-7.84259 4.578829-6.694894C4.578829-6.264508 4.542964-5.66675 4.196264-4.327771H1.673724ZM4.112578-3.969116C3.813699-2.797509 3.56264-2.044334 3.132254-1.291158C2.785554-.681445 2.367123-.119552 1.865006-.119552C1.494396-.119552 1.195517-.406476 1.195517-1.590037C1.195517-2.367123 1.3868-3.180075 1.578082-3.969116H4.112578Z'/> +<path id='g0-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/> +<path id='g0-86' d='M7.400249-6.838356C7.806725-7.483935 8.177335-7.770859 8.787049-7.81868C8.9066-7.830635 9.002242-7.830635 9.002242-8.045828C9.002242-8.093649 8.978331-8.16538 8.870735-8.16538C8.655542-8.16538 8.141469-8.141469 7.926276-8.141469C7.579577-8.141469 7.220922-8.16538 6.886177-8.16538C6.790535-8.16538 6.670984-8.16538 6.670984-7.938232C6.670984-7.830635 6.77858-7.81868 6.826401-7.81868C7.268742-7.782814 7.316563-7.567621 7.316563-7.424159C7.316563-7.244832 7.149191-6.969863 7.137235-6.957908L3.383313-1.004234L2.546451-7.44807C2.546451-7.79477 3.16812-7.81868 3.299626-7.81868C3.478954-7.81868 3.58655-7.81868 3.58655-8.045828C3.58655-8.16538 3.455044-8.16538 3.419178-8.16538C3.21594-8.16538 2.976837-8.141469 2.773599-8.141469H2.10411C1.231382-8.141469 .872727-8.16538 .860772-8.16538C.789041-8.16538 .645579-8.16538 .645579-7.950187C.645579-7.81868 .729265-7.81868 .920548-7.81868C1.530262-7.81868 1.566127-7.711083 1.601993-7.412204L2.558406-.035866C2.594271 .215193 2.594271 .251059 2.761644 .251059C2.905106 .251059 2.964882 .215193 3.084433 .02391L7.400249-6.838356Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -65.03376)'> +<use x='56.413267' y='65.753425' xlink:href='#g0-86'/> +<use x='63.237617' y='67.546688' xlink:href='#g1-49'/> +<use x='71.290761' y='65.753425' xlink:href='#g2-61'/> +<use x='83.716242' y='65.753425' xlink:href='#g2-48'/> +<use x='89.569232' y='65.753425' xlink:href='#g0-58'/> +<use x='92.820894' y='65.753425' xlink:href='#g2-48'/> +<use x='98.673884' y='65.753425' xlink:href='#g2-50'/> +<use x='104.526874' y='65.753425' xlink:href='#g0-18'/> +<use x='109.982049' y='67.546688' xlink:href='#g1-49'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/bias-c2.svg b/_resources/bias-c2.svg new file mode 100644 index 0000000..c2cacfc --- /dev/null +++ b/_resources/bias-c2.svg @@ -0,0 +1,34 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='75.975638pt' height='28.200187pt' viewBox='-.244117 -.227486 75.975638 28.200187'> +<defs> +<path id='g0-0' d='M7.878456-2.749689C8.081694-2.749689 8.296887-2.749689 8.296887-2.988792S8.081694-3.227895 7.878456-3.227895H1.41071C1.207472-3.227895 .992279-3.227895 .992279-2.988792S1.207472-2.749689 1.41071-2.749689H7.878456Z'/> +<path id='g1-18' d='M5.296139-6.01345C5.296139-7.232877 4.913574-8.416438 3.93325-8.416438C2.259527-8.416438 .478207-4.913574 .478207-2.283437C.478207-1.733499 .597758 .119552 1.853051 .119552C3.478954 .119552 5.296139-3.299626 5.296139-6.01345ZM1.673724-4.327771C1.853051-5.033126 2.10411-6.03736 2.582316-6.886177C2.976837-7.603487 3.395268-8.177335 3.921295-8.177335C4.315816-8.177335 4.578829-7.84259 4.578829-6.694894C4.578829-6.264508 4.542964-5.66675 4.196264-4.327771H1.673724ZM4.112578-3.969116C3.813699-2.797509 3.56264-2.044334 3.132254-1.291158C2.785554-.681445 2.367123-.119552 1.865006-.119552C1.494396-.119552 1.195517-.406476 1.195517-1.590037C1.195517-2.367123 1.3868-3.180075 1.578082-3.969116H4.112578Z'/> +<path id='g1-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/> +<path id='g2-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g3-48' d='M5.355915-3.825654C5.355915-4.817933 5.296139-5.786301 4.865753-6.694894C4.375592-7.687173 3.514819-7.950187 2.929016-7.950187C2.235616-7.950187 1.3868-7.603487 .944458-6.611208C.609714-5.858032 .490162-5.116812 .490162-3.825654C.490162-2.666002 .573848-1.793275 1.004234-.944458C1.470486-.035866 2.295392 .251059 2.917061 .251059C3.957161 .251059 4.554919-.37061 4.901619-1.06401C5.332005-1.960648 5.355915-3.132254 5.355915-3.825654ZM2.917061 .011955C2.534496 .011955 1.75741-.203238 1.530262-1.506351C1.398755-2.223661 1.398755-3.132254 1.398755-3.969116C1.398755-4.94944 1.398755-5.834122 1.590037-6.539477C1.793275-7.340473 2.402989-7.711083 2.917061-7.711083C3.371357-7.711083 4.064757-7.436115 4.291905-6.40797C4.447323-5.726526 4.447323-4.782067 4.447323-3.969116C4.447323-3.16812 4.447323-2.259527 4.315816-1.530262C4.088667-.215193 3.335492 .011955 2.917061 .011955Z'/> +<path id='g3-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +<path id='g3-50' d='M5.260274-2.008468H4.99726C4.961395-1.80523 4.865753-1.147696 4.746202-.956413C4.662516-.848817 3.981071-.848817 3.622416-.848817H1.41071C1.733499-1.123786 2.462765-1.888917 2.773599-2.175841C4.590785-3.849564 5.260274-4.471233 5.260274-5.654795C5.260274-7.029639 4.172354-7.950187 2.785554-7.950187S.585803-6.766625 .585803-5.738481C.585803-5.128767 1.111831-5.128767 1.147696-5.128767C1.398755-5.128767 1.709589-5.308095 1.709589-5.69066C1.709589-6.025405 1.482441-6.252553 1.147696-6.252553C1.0401-6.252553 1.016189-6.252553 .980324-6.240598C1.207472-7.053549 1.853051-7.603487 2.630137-7.603487C3.646326-7.603487 4.267995-6.75467 4.267995-5.654795C4.267995-4.638605 3.682192-3.753923 3.000747-2.988792L.585803-.286924V0H4.94944L5.260274-2.008468Z'/> +<path id='g3-56' d='M3.56264-4.315816C4.160399-4.638605 5.033126-5.188543 5.033126-6.192777C5.033126-7.232877 4.028892-7.950187 2.929016-7.950187C1.745455-7.950187 .812951-7.07746 .812951-5.989539C.812951-5.583064 .932503-5.176588 1.267248-4.770112C1.398755-4.614695 1.41071-4.60274 2.247572-4.016936C1.08792-3.478954 .490162-2.677958 .490162-1.80523C.490162-.537983 1.697634 .251059 2.917061 .251059C4.244085 .251059 5.355915-.729265 5.355915-1.984558C5.355915-3.203985 4.495143-3.741968 3.56264-4.315816ZM1.936737-5.391781C1.78132-5.499377 1.303113-5.810212 1.303113-6.396015C1.303113-7.173101 2.116065-7.663263 2.917061-7.663263C3.777833-7.663263 4.542964-7.041594 4.542964-6.180822C4.542964-5.451557 4.016936-4.865753 3.323537-4.483188L1.936737-5.391781ZM2.49863-3.849564L3.945205-2.905106C4.25604-2.701868 4.805978-2.331258 4.805978-1.601993C4.805978-.6934 3.88543-.071731 2.929016-.071731C1.912827-.071731 1.0401-.812951 1.0401-1.80523C1.0401-2.737733 1.721544-3.490909 2.49863-3.849564Z'/> +<path id='g3-57' d='M4.375592-3.478954C4.375592-.657534 3.120299-.071731 2.402989-.071731C2.116065-.071731 1.482441-.107597 1.183562-.526027H1.255293C1.338979-.502117 1.769365-.573848 1.769365-1.016189C1.769365-1.279203 1.590037-1.506351 1.279203-1.506351S.777086-1.303113 .777086-.992279C.777086-.251059 1.374844 .251059 2.414944 .251059C3.90934 .251059 5.355915-1.338979 5.355915-3.93325C5.355915-7.149191 4.016936-7.950187 2.964882-7.950187C1.649813-7.950187 .490162-6.850311 .490162-5.272229S1.601993-2.618182 2.797509-2.618182C3.682192-2.618182 4.136488-3.263761 4.375592-3.873474V-3.478954ZM2.84533-2.857285C2.092154-2.857285 1.769365-3.466999 1.661768-3.694147C1.470486-4.148443 1.470486-4.722291 1.470486-5.260274C1.470486-5.929763 1.470486-6.503611 1.78132-6.993773C1.996513-7.316563 2.319303-7.663263 2.964882-7.663263C3.646326-7.663263 3.993026-7.065504 4.112578-6.790535C4.351681-6.204732 4.351681-5.188543 4.351681-5.009215C4.351681-4.004981 3.897385-2.857285 2.84533-2.857285Z'/> +<path id='g3-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -65.34204 -60.890404)'> +<use x='62.747439' y='62.100436' xlink:href='#g3-48'/> +<use x='68.60043' y='62.100436' xlink:href='#g1-58'/> +<use x='71.852091' y='62.100436' xlink:href='#g3-48'/> +<use x='77.705081' y='62.100436' xlink:href='#g3-50'/> +<use x='83.558071' y='62.100436' xlink:href='#g1-18'/> +<use x='89.013246' y='63.893699' xlink:href='#g2-49'/> +<rect x='57.608781' y='66.960309' height='.478187' width='41.275439'/> +<use x='57.608781' y='78.388857' xlink:href='#g3-49'/> +<use x='66.118435' y='78.388857' xlink:href='#g0-0'/> +<use x='78.073595' y='78.388857' xlink:href='#g3-48'/> +<use x='83.926585' y='78.388857' xlink:href='#g1-58'/> +<use x='87.178247' y='78.388857' xlink:href='#g3-57'/> +<use x='93.031237' y='78.388857' xlink:href='#g3-56'/> +<use x='103.400563' y='70.188195' xlink:href='#g3-61'/> +<use x='115.826044' y='70.188195' xlink:href='#g1-18'/> +<use x='121.281218' y='71.981458' xlink:href='#g2-49'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/bias-c3.svg b/_resources/bias-c3.svg new file mode 100644 index 0000000..90262fc --- /dev/null +++ b/_resources/bias-c3.svg @@ -0,0 +1,38 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='129.376645pt' height='11.536963pt' viewBox='-.239051 -.242965 129.376645 11.536963'> +<defs> +<path id='g2-43' d='M4.770112-2.761644H8.069738C8.237111-2.761644 8.452304-2.761644 8.452304-2.976837C8.452304-3.203985 8.249066-3.203985 8.069738-3.203985H4.770112V-6.503611C4.770112-6.670984 4.770112-6.886177 4.554919-6.886177C4.327771-6.886177 4.327771-6.682939 4.327771-6.503611V-3.203985H1.028144C.860772-3.203985 .645579-3.203985 .645579-2.988792C.645579-2.761644 .848817-2.761644 1.028144-2.761644H4.327771V.537983C4.327771 .705355 4.327771 .920548 4.542964 .920548C4.770112 .920548 4.770112 .71731 4.770112 .537983V-2.761644Z'/> +<path id='g2-48' d='M5.355915-3.825654C5.355915-4.817933 5.296139-5.786301 4.865753-6.694894C4.375592-7.687173 3.514819-7.950187 2.929016-7.950187C2.235616-7.950187 1.3868-7.603487 .944458-6.611208C.609714-5.858032 .490162-5.116812 .490162-3.825654C.490162-2.666002 .573848-1.793275 1.004234-.944458C1.470486-.035866 2.295392 .251059 2.917061 .251059C3.957161 .251059 4.554919-.37061 4.901619-1.06401C5.332005-1.960648 5.355915-3.132254 5.355915-3.825654ZM2.917061 .011955C2.534496 .011955 1.75741-.203238 1.530262-1.506351C1.398755-2.223661 1.398755-3.132254 1.398755-3.969116C1.398755-4.94944 1.398755-5.834122 1.590037-6.539477C1.793275-7.340473 2.402989-7.711083 2.917061-7.711083C3.371357-7.711083 4.064757-7.436115 4.291905-6.40797C4.447323-5.726526 4.447323-4.782067 4.447323-3.969116C4.447323-3.16812 4.447323-2.259527 4.315816-1.530262C4.088667-.215193 3.335492 .011955 2.917061 .011955Z'/> +<path id='g2-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +<path id='g2-50' d='M5.260274-2.008468H4.99726C4.961395-1.80523 4.865753-1.147696 4.746202-.956413C4.662516-.848817 3.981071-.848817 3.622416-.848817H1.41071C1.733499-1.123786 2.462765-1.888917 2.773599-2.175841C4.590785-3.849564 5.260274-4.471233 5.260274-5.654795C5.260274-7.029639 4.172354-7.950187 2.785554-7.950187S.585803-6.766625 .585803-5.738481C.585803-5.128767 1.111831-5.128767 1.147696-5.128767C1.398755-5.128767 1.709589-5.308095 1.709589-5.69066C1.709589-6.025405 1.482441-6.252553 1.147696-6.252553C1.0401-6.252553 1.016189-6.252553 .980324-6.240598C1.207472-7.053549 1.853051-7.603487 2.630137-7.603487C3.646326-7.603487 4.267995-6.75467 4.267995-5.654795C4.267995-4.638605 3.682192-3.753923 3.000747-2.988792L.585803-.286924V0H4.94944L5.260274-2.008468Z'/> +<path id='g2-54' d='M1.470486-4.160399C1.470486-7.185056 2.940971-7.663263 3.58655-7.663263C4.016936-7.663263 4.447323-7.531756 4.674471-7.173101C4.531009-7.173101 4.076712-7.173101 4.076712-6.682939C4.076712-6.419925 4.25604-6.192777 4.566874-6.192777C4.865753-6.192777 5.068991-6.372105 5.068991-6.718804C5.068991-7.340473 4.614695-7.950187 3.574595-7.950187C2.068244-7.950187 .490162-6.40797 .490162-3.777833C.490162-.490162 1.924782 .251059 2.940971 .251059C4.244085 .251059 5.355915-.884682 5.355915-2.438854C5.355915-4.028892 4.244085-5.092902 3.048568-5.092902C1.984558-5.092902 1.590037-4.172354 1.470486-3.837609V-4.160399ZM2.940971-.071731C2.187796-.071731 1.829141-.74122 1.721544-.992279C1.613948-1.303113 1.494396-1.888917 1.494396-2.725778C1.494396-3.670237 1.924782-4.853798 3.000747-4.853798C3.658281-4.853798 4.004981-4.411457 4.184309-4.004981C4.375592-3.56264 4.375592-2.964882 4.375592-2.450809C4.375592-1.841096 4.375592-1.303113 4.148443-.848817C3.849564-.274969 3.419178-.071731 2.940971-.071731Z'/> +<path id='g2-57' d='M4.375592-3.478954C4.375592-.657534 3.120299-.071731 2.402989-.071731C2.116065-.071731 1.482441-.107597 1.183562-.526027H1.255293C1.338979-.502117 1.769365-.573848 1.769365-1.016189C1.769365-1.279203 1.590037-1.506351 1.279203-1.506351S.777086-1.303113 .777086-.992279C.777086-.251059 1.374844 .251059 2.414944 .251059C3.90934 .251059 5.355915-1.338979 5.355915-3.93325C5.355915-7.149191 4.016936-7.950187 2.964882-7.950187C1.649813-7.950187 .490162-6.850311 .490162-5.272229S1.601993-2.618182 2.797509-2.618182C3.682192-2.618182 4.136488-3.263761 4.375592-3.873474V-3.478954ZM2.84533-2.857285C2.092154-2.857285 1.769365-3.466999 1.661768-3.694147C1.470486-4.148443 1.470486-4.722291 1.470486-5.260274C1.470486-5.929763 1.470486-6.503611 1.78132-6.993773C1.996513-7.316563 2.319303-7.663263 2.964882-7.663263C3.646326-7.663263 3.993026-7.065504 4.112578-6.790535C4.351681-6.204732 4.351681-5.188543 4.351681-5.009215C4.351681-4.004981 3.897385-2.857285 2.84533-2.857285Z'/> +<path id='g2-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g1-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g1-50' d='M2.247572-1.625903C2.375093-1.745455 2.709838-2.008468 2.83736-2.12005C3.331507-2.574346 3.801743-3.012702 3.801743-3.737983C3.801743-4.686426 3.004732-5.300125 2.008468-5.300125C1.052055-5.300125 .422416-4.574844 .422416-3.865504C.422416-3.474969 .73325-3.419178 .844832-3.419178C1.012204-3.419178 1.259278-3.53873 1.259278-3.841594C1.259278-4.25604 .860772-4.25604 .765131-4.25604C.996264-4.837858 1.530262-5.037111 1.920797-5.037111C2.662017-5.037111 3.044583-4.407472 3.044583-3.737983C3.044583-2.909091 2.462765-2.303362 1.522291-1.338979L.518057-.302864C.422416-.215193 .422416-.199253 .422416 0H3.57061L3.801743-1.42665H3.55467C3.53076-1.267248 3.466999-.868742 3.371357-.71731C3.323537-.653549 2.717808-.653549 2.590286-.653549H1.171606L2.247572-1.625903Z'/> +<path id='g0-18' d='M5.296139-6.01345C5.296139-7.232877 4.913574-8.416438 3.93325-8.416438C2.259527-8.416438 .478207-4.913574 .478207-2.283437C.478207-1.733499 .597758 .119552 1.853051 .119552C3.478954 .119552 5.296139-3.299626 5.296139-6.01345ZM1.673724-4.327771C1.853051-5.033126 2.10411-6.03736 2.582316-6.886177C2.976837-7.603487 3.395268-8.177335 3.921295-8.177335C4.315816-8.177335 4.578829-7.84259 4.578829-6.694894C4.578829-6.264508 4.542964-5.66675 4.196264-4.327771H1.673724ZM4.112578-3.969116C3.813699-2.797509 3.56264-2.044334 3.132254-1.291158C2.785554-.681445 2.367123-.119552 1.865006-.119552C1.494396-.119552 1.195517-.406476 1.195517-1.590037C1.195517-2.367123 1.3868-3.180075 1.578082-3.969116H4.112578Z'/> +<path id='g0-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/> +<path id='g0-86' d='M7.400249-6.838356C7.806725-7.483935 8.177335-7.770859 8.787049-7.81868C8.9066-7.830635 9.002242-7.830635 9.002242-8.045828C9.002242-8.093649 8.978331-8.16538 8.870735-8.16538C8.655542-8.16538 8.141469-8.141469 7.926276-8.141469C7.579577-8.141469 7.220922-8.16538 6.886177-8.16538C6.790535-8.16538 6.670984-8.16538 6.670984-7.938232C6.670984-7.830635 6.77858-7.81868 6.826401-7.81868C7.268742-7.782814 7.316563-7.567621 7.316563-7.424159C7.316563-7.244832 7.149191-6.969863 7.137235-6.957908L3.383313-1.004234L2.546451-7.44807C2.546451-7.79477 3.16812-7.81868 3.299626-7.81868C3.478954-7.81868 3.58655-7.81868 3.58655-8.045828C3.58655-8.16538 3.455044-8.16538 3.419178-8.16538C3.21594-8.16538 2.976837-8.141469 2.773599-8.141469H2.10411C1.231382-8.141469 .872727-8.16538 .860772-8.16538C.789041-8.16538 .645579-8.16538 .645579-7.950187C.645579-7.81868 .729265-7.81868 .920548-7.81868C1.530262-7.81868 1.566127-7.711083 1.601993-7.412204L2.558406-.035866C2.594271 .215193 2.594271 .251059 2.761644 .251059C2.905106 .251059 2.964882 .215193 3.084433 .02391L7.400249-6.838356Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -65.03376)'> +<use x='56.413267' y='65.753425' xlink:href='#g0-86'/> +<use x='63.237617' y='67.546688' xlink:href='#g1-50'/> +<use x='71.290761' y='65.753425' xlink:href='#g2-61'/> +<use x='83.716242' y='65.753425' xlink:href='#g2-48'/> +<use x='89.569232' y='65.753425' xlink:href='#g0-58'/> +<use x='92.820894' y='65.753425' xlink:href='#g2-48'/> +<use x='98.673884' y='65.753425' xlink:href='#g2-49'/> +<use x='104.526874' y='65.753425' xlink:href='#g2-57'/> +<use x='110.379864' y='65.753425' xlink:href='#g2-54'/> +<use x='116.232855' y='65.753425' xlink:href='#g0-18'/> +<use x='121.688029' y='67.546688' xlink:href='#g1-49'/> +<use x='129.077008' y='65.753425' xlink:href='#g2-43'/> +<use x='140.838323' y='65.753425' xlink:href='#g2-48'/> +<use x='146.691313' y='65.753425' xlink:href='#g0-58'/> +<use x='149.942974' y='65.753425' xlink:href='#g2-48'/> +<use x='155.795965' y='65.753425' xlink:href='#g2-50'/> +<use x='161.648955' y='65.753425' xlink:href='#g0-18'/> +<use x='167.10413' y='67.546688' xlink:href='#g1-50'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/bias-c4.svg b/_resources/bias-c4.svg new file mode 100644 index 0000000..ebc6b67 --- /dev/null +++ b/_resources/bias-c4.svg @@ -0,0 +1,68 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='219.646849pt' height='28.200187pt' viewBox='-.244117 -.227486 219.646849 28.200187'> +<defs> +<path id='g0-0' d='M7.878456-2.749689C8.081694-2.749689 8.296887-2.749689 8.296887-2.988792S8.081694-3.227895 7.878456-3.227895H1.41071C1.207472-3.227895 .992279-3.227895 .992279-2.988792S1.207472-2.749689 1.41071-2.749689H7.878456Z'/> +<path id='g2-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g2-50' d='M2.247572-1.625903C2.375093-1.745455 2.709838-2.008468 2.83736-2.12005C3.331507-2.574346 3.801743-3.012702 3.801743-3.737983C3.801743-4.686426 3.004732-5.300125 2.008468-5.300125C1.052055-5.300125 .422416-4.574844 .422416-3.865504C.422416-3.474969 .73325-3.419178 .844832-3.419178C1.012204-3.419178 1.259278-3.53873 1.259278-3.841594C1.259278-4.25604 .860772-4.25604 .765131-4.25604C.996264-4.837858 1.530262-5.037111 1.920797-5.037111C2.662017-5.037111 3.044583-4.407472 3.044583-3.737983C3.044583-2.909091 2.462765-2.303362 1.522291-1.338979L.518057-.302864C.422416-.215193 .422416-.199253 .422416 0H3.57061L3.801743-1.42665H3.55467C3.53076-1.267248 3.466999-.868742 3.371357-.71731C3.323537-.653549 2.717808-.653549 2.590286-.653549H1.171606L2.247572-1.625903Z'/> +<path id='g1-18' d='M5.296139-6.01345C5.296139-7.232877 4.913574-8.416438 3.93325-8.416438C2.259527-8.416438 .478207-4.913574 .478207-2.283437C.478207-1.733499 .597758 .119552 1.853051 .119552C3.478954 .119552 5.296139-3.299626 5.296139-6.01345ZM1.673724-4.327771C1.853051-5.033126 2.10411-6.03736 2.582316-6.886177C2.976837-7.603487 3.395268-8.177335 3.921295-8.177335C4.315816-8.177335 4.578829-7.84259 4.578829-6.694894C4.578829-6.264508 4.542964-5.66675 4.196264-4.327771H1.673724ZM4.112578-3.969116C3.813699-2.797509 3.56264-2.044334 3.132254-1.291158C2.785554-.681445 2.367123-.119552 1.865006-.119552C1.494396-.119552 1.195517-.406476 1.195517-1.590037C1.195517-2.367123 1.3868-3.180075 1.578082-3.969116H4.112578Z'/> +<path id='g1-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/> +<path id='g3-43' d='M4.770112-2.761644H8.069738C8.237111-2.761644 8.452304-2.761644 8.452304-2.976837C8.452304-3.203985 8.249066-3.203985 8.069738-3.203985H4.770112V-6.503611C4.770112-6.670984 4.770112-6.886177 4.554919-6.886177C4.327771-6.886177 4.327771-6.682939 4.327771-6.503611V-3.203985H1.028144C.860772-3.203985 .645579-3.203985 .645579-2.988792C.645579-2.761644 .848817-2.761644 1.028144-2.761644H4.327771V.537983C4.327771 .705355 4.327771 .920548 4.542964 .920548C4.770112 .920548 4.770112 .71731 4.770112 .537983V-2.761644Z'/> +<path id='g3-48' d='M5.355915-3.825654C5.355915-4.817933 5.296139-5.786301 4.865753-6.694894C4.375592-7.687173 3.514819-7.950187 2.929016-7.950187C2.235616-7.950187 1.3868-7.603487 .944458-6.611208C.609714-5.858032 .490162-5.116812 .490162-3.825654C.490162-2.666002 .573848-1.793275 1.004234-.944458C1.470486-.035866 2.295392 .251059 2.917061 .251059C3.957161 .251059 4.554919-.37061 4.901619-1.06401C5.332005-1.960648 5.355915-3.132254 5.355915-3.825654ZM2.917061 .011955C2.534496 .011955 1.75741-.203238 1.530262-1.506351C1.398755-2.223661 1.398755-3.132254 1.398755-3.969116C1.398755-4.94944 1.398755-5.834122 1.590037-6.539477C1.793275-7.340473 2.402989-7.711083 2.917061-7.711083C3.371357-7.711083 4.064757-7.436115 4.291905-6.40797C4.447323-5.726526 4.447323-4.782067 4.447323-3.969116C4.447323-3.16812 4.447323-2.259527 4.315816-1.530262C4.088667-.215193 3.335492 .011955 2.917061 .011955Z'/> +<path id='g3-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +<path id='g3-50' d='M5.260274-2.008468H4.99726C4.961395-1.80523 4.865753-1.147696 4.746202-.956413C4.662516-.848817 3.981071-.848817 3.622416-.848817H1.41071C1.733499-1.123786 2.462765-1.888917 2.773599-2.175841C4.590785-3.849564 5.260274-4.471233 5.260274-5.654795C5.260274-7.029639 4.172354-7.950187 2.785554-7.950187S.585803-6.766625 .585803-5.738481C.585803-5.128767 1.111831-5.128767 1.147696-5.128767C1.398755-5.128767 1.709589-5.308095 1.709589-5.69066C1.709589-6.025405 1.482441-6.252553 1.147696-6.252553C1.0401-6.252553 1.016189-6.252553 .980324-6.240598C1.207472-7.053549 1.853051-7.603487 2.630137-7.603487C3.646326-7.603487 4.267995-6.75467 4.267995-5.654795C4.267995-4.638605 3.682192-3.753923 3.000747-2.988792L.585803-.286924V0H4.94944L5.260274-2.008468Z'/> +<path id='g3-51' d='M2.199751-4.291905C1.996513-4.27995 1.948692-4.267995 1.948692-4.160399C1.948692-4.040847 2.008468-4.040847 2.223661-4.040847H2.773599C3.789788-4.040847 4.244085-3.203985 4.244085-2.056289C4.244085-.490162 3.431133-.071731 2.84533-.071731C2.271482-.071731 1.291158-.3467 .944458-1.135741C1.327024-1.075965 1.673724-1.291158 1.673724-1.721544C1.673724-2.068244 1.422665-2.307347 1.08792-2.307347C.800996-2.307347 .490162-2.139975 .490162-1.685679C.490162-.621669 1.554172 .251059 2.881196 .251059C4.303861 .251059 5.355915-.836862 5.355915-2.044334C5.355915-3.144209 4.471233-4.004981 3.323537-4.208219C4.363636-4.507098 5.033126-5.379826 5.033126-6.312329C5.033126-7.256787 4.052802-7.950187 2.893151-7.950187C1.697634-7.950187 .812951-7.220922 .812951-6.348194C.812951-5.869988 1.183562-5.774346 1.362889-5.774346C1.613948-5.774346 1.900872-5.953674 1.900872-6.312329C1.900872-6.694894 1.613948-6.862267 1.350934-6.862267C1.279203-6.862267 1.255293-6.862267 1.219427-6.850311C1.673724-7.663263 2.797509-7.663263 2.857285-7.663263C3.251806-7.663263 4.028892-7.483935 4.028892-6.312329C4.028892-6.085181 3.993026-5.415691 3.646326-4.901619C3.287671-4.375592 2.881196-4.339726 2.558406-4.327771L2.199751-4.291905Z'/> +<path id='g3-54' d='M1.470486-4.160399C1.470486-7.185056 2.940971-7.663263 3.58655-7.663263C4.016936-7.663263 4.447323-7.531756 4.674471-7.173101C4.531009-7.173101 4.076712-7.173101 4.076712-6.682939C4.076712-6.419925 4.25604-6.192777 4.566874-6.192777C4.865753-6.192777 5.068991-6.372105 5.068991-6.718804C5.068991-7.340473 4.614695-7.950187 3.574595-7.950187C2.068244-7.950187 .490162-6.40797 .490162-3.777833C.490162-.490162 1.924782 .251059 2.940971 .251059C4.244085 .251059 5.355915-.884682 5.355915-2.438854C5.355915-4.028892 4.244085-5.092902 3.048568-5.092902C1.984558-5.092902 1.590037-4.172354 1.470486-3.837609V-4.160399ZM2.940971-.071731C2.187796-.071731 1.829141-.74122 1.721544-.992279C1.613948-1.303113 1.494396-1.888917 1.494396-2.725778C1.494396-3.670237 1.924782-4.853798 3.000747-4.853798C3.658281-4.853798 4.004981-4.411457 4.184309-4.004981C4.375592-3.56264 4.375592-2.964882 4.375592-2.450809C4.375592-1.841096 4.375592-1.303113 4.148443-.848817C3.849564-.274969 3.419178-.071731 2.940971-.071731Z'/> +<path id='g3-56' d='M3.56264-4.315816C4.160399-4.638605 5.033126-5.188543 5.033126-6.192777C5.033126-7.232877 4.028892-7.950187 2.929016-7.950187C1.745455-7.950187 .812951-7.07746 .812951-5.989539C.812951-5.583064 .932503-5.176588 1.267248-4.770112C1.398755-4.614695 1.41071-4.60274 2.247572-4.016936C1.08792-3.478954 .490162-2.677958 .490162-1.80523C.490162-.537983 1.697634 .251059 2.917061 .251059C4.244085 .251059 5.355915-.729265 5.355915-1.984558C5.355915-3.203985 4.495143-3.741968 3.56264-4.315816ZM1.936737-5.391781C1.78132-5.499377 1.303113-5.810212 1.303113-6.396015C1.303113-7.173101 2.116065-7.663263 2.917061-7.663263C3.777833-7.663263 4.542964-7.041594 4.542964-6.180822C4.542964-5.451557 4.016936-4.865753 3.323537-4.483188L1.936737-5.391781ZM2.49863-3.849564L3.945205-2.905106C4.25604-2.701868 4.805978-2.331258 4.805978-1.601993C4.805978-.6934 3.88543-.071731 2.929016-.071731C1.912827-.071731 1.0401-.812951 1.0401-1.80523C1.0401-2.737733 1.721544-3.490909 2.49863-3.849564Z'/> +<path id='g3-57' d='M4.375592-3.478954C4.375592-.657534 3.120299-.071731 2.402989-.071731C2.116065-.071731 1.482441-.107597 1.183562-.526027H1.255293C1.338979-.502117 1.769365-.573848 1.769365-1.016189C1.769365-1.279203 1.590037-1.506351 1.279203-1.506351S.777086-1.303113 .777086-.992279C.777086-.251059 1.374844 .251059 2.414944 .251059C3.90934 .251059 5.355915-1.338979 5.355915-3.93325C5.355915-7.149191 4.016936-7.950187 2.964882-7.950187C1.649813-7.950187 .490162-6.850311 .490162-5.272229S1.601993-2.618182 2.797509-2.618182C3.682192-2.618182 4.136488-3.263761 4.375592-3.873474V-3.478954ZM2.84533-2.857285C2.092154-2.857285 1.769365-3.466999 1.661768-3.694147C1.470486-4.148443 1.470486-4.722291 1.470486-5.260274C1.470486-5.929763 1.470486-6.503611 1.78132-6.993773C1.996513-7.316563 2.319303-7.663263 2.964882-7.663263C3.646326-7.663263 3.993026-7.065504 4.112578-6.790535C4.351681-6.204732 4.351681-5.188543 4.351681-5.009215C4.351681-4.004981 3.897385-2.857285 2.84533-2.857285Z'/> +<path id='g3-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -65.34204 -60.890404)'> +<use x='57.608781' y='62.100436' xlink:href='#g3-48'/> +<use x='63.461771' y='62.100436' xlink:href='#g1-58'/> +<use x='66.713432' y='62.100436' xlink:href='#g3-48'/> +<use x='72.566423' y='62.100436' xlink:href='#g3-49'/> +<use x='78.419413' y='62.100436' xlink:href='#g3-57'/> +<use x='84.272403' y='62.100436' xlink:href='#g3-54'/> +<use x='90.125393' y='62.100436' xlink:href='#g1-18'/> +<use x='95.580568' y='63.893699' xlink:href='#g2-49'/> +<use x='102.969547' y='62.100436' xlink:href='#g3-43'/> +<use x='114.730862' y='62.100436' xlink:href='#g3-48'/> +<use x='120.583852' y='62.100436' xlink:href='#g1-58'/> +<use x='123.835513' y='62.100436' xlink:href='#g3-48'/> +<use x='129.688503' y='62.100436' xlink:href='#g3-50'/> +<use x='135.541494' y='62.100436' xlink:href='#g1-18'/> +<use x='140.996668' y='63.893699' xlink:href='#g2-50'/> +<rect x='57.608781' y='66.960309' height='.478187' width='88.120202'/> +<use x='78.665013' y='78.388857' xlink:href='#g3-49'/> +<use x='87.174667' y='78.388857' xlink:href='#g0-0'/> +<use x='99.129827' y='78.388857' xlink:href='#g3-48'/> +<use x='104.982817' y='78.388857' xlink:href='#g1-58'/> +<use x='108.234479' y='78.388857' xlink:href='#g3-57'/> +<use x='114.087469' y='78.388857' xlink:href='#g3-56'/> +<use x='119.940459' y='74.935149' xlink:href='#g2-50'/> +<use x='150.245326' y='70.188195' xlink:href='#g3-61'/> +<use x='163.866321' y='62.100436' xlink:href='#g3-48'/> +<use x='169.719311' y='62.100436' xlink:href='#g1-58'/> +<use x='172.970973' y='62.100436' xlink:href='#g3-48'/> +<use x='178.823963' y='62.100436' xlink:href='#g3-49'/> +<use x='184.676953' y='62.100436' xlink:href='#g3-57'/> +<use x='190.529943' y='62.100436' xlink:href='#g3-54'/> +<use x='196.382934' y='62.100436' xlink:href='#g1-18'/> +<use x='201.838108' y='63.893699' xlink:href='#g2-49'/> +<use x='209.227087' y='62.100436' xlink:href='#g3-43'/> +<use x='220.988402' y='62.100436' xlink:href='#g3-48'/> +<use x='226.841392' y='62.100436' xlink:href='#g1-58'/> +<use x='230.093053' y='62.100436' xlink:href='#g3-48'/> +<use x='235.946043' y='62.100436' xlink:href='#g3-50'/> +<use x='241.799034' y='62.100436' xlink:href='#g1-18'/> +<use x='247.254208' y='63.893699' xlink:href='#g2-50'/> +<rect x='163.866321' y='66.960309' height='.478187' width='88.120202'/> +<use x='191.668116' y='78.388857' xlink:href='#g3-48'/> +<use x='197.521106' y='78.388857' xlink:href='#g1-58'/> +<use x='200.772767' y='78.388857' xlink:href='#g3-48'/> +<use x='206.625758' y='78.388857' xlink:href='#g3-51'/> +<use x='212.478748' y='78.388857' xlink:href='#g3-57'/> +<use x='218.331738' y='78.388857' xlink:href='#g3-54'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/bias-variance-mismatch-1.png b/_resources/bias-variance-mismatch-1.png new file mode 100644 index 0000000..0d9114f Binary files /dev/null and b/_resources/bias-variance-mismatch-1.png differ diff --git a/_resources/bias-variance-mismatch.png b/_resources/bias-variance-mismatch.png new file mode 100644 index 0000000..3ffb8ea Binary files /dev/null and b/_resources/bias-variance-mismatch.png differ diff --git a/_resources/bias-variance-tradeoff.png b/_resources/bias-variance-tradeoff.png new file mode 100644 index 0000000..0d793a1 Binary files /dev/null and b/_resources/bias-variance-tradeoff.png differ diff --git a/_resources/bleu-combined.svg b/_resources/bleu-combined.svg new file mode 100644 index 0000000..16a109d --- /dev/null +++ b/_resources/bleu-combined.svg @@ -0,0 +1,37 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='106.428817pt' height='22.176757pt' viewBox='-.299738 -.244857 106.428817 22.176757'> +<defs> +<path id='g1-110' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.833126-2.279452 1.833126-2.287422 2.016438-2.550436C2.279452-2.940971 2.654047-3.291656 3.188045-3.291656C3.474969-3.291656 3.642341-3.124284 3.642341-2.749689C3.642341-2.311333 3.307597-1.40274 3.156164-1.012204C3.052553-.749191 3.052553-.70137 3.052553-.597758C3.052553-.143462 3.427148 .079701 3.769863 .079701C4.550934 .079701 4.877709-1.036115 4.877709-1.139726C4.877709-1.219427 4.813948-1.243337 4.758157-1.243337C4.662516-1.243337 4.646575-1.187547 4.622665-1.107846C4.431382-.454296 4.096638-.143462 3.793773-.143462C3.666252-.143462 3.602491-.223163 3.602491-.406476S3.666252-.765131 3.745953-.964384C3.865504-1.267248 4.216189-2.183811 4.216189-2.630137C4.216189-3.227895 3.801743-3.514819 3.227895-3.514819C2.582316-3.514819 2.16787-3.124284 1.936737-2.82142C1.880946-3.259776 1.530262-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.318804-2.709838 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g3-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g3-52' d='M3.140224-5.156663C3.140224-5.316065 3.140224-5.379826 2.972852-5.379826C2.86924-5.379826 2.86127-5.371856 2.781569-5.260274L.239103-1.570112V-1.307098H2.486675V-.645579C2.486675-.350685 2.462765-.263014 1.849066-.263014H1.665753V0C2.343213-.02391 2.359153-.02391 2.81345-.02391S3.283686-.02391 3.961146 0V-.263014H3.777833C3.164134-.263014 3.140224-.350685 3.140224-.645579V-1.307098H3.985056V-1.570112H3.140224V-5.156663ZM2.542466-4.511083V-1.570112H.518057L2.542466-4.511083Z'/> +<path id='g3-61' d='M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z'/> +<path id='g0-16' d='M5.61818 19.069086C5.639998 19.079995 5.73818 19.189086 5.749089 19.189086H5.989089C6.021817 19.189086 6.109089 19.178176 6.109089 19.079995C6.109089 19.036358 6.087271 19.01454 6.065453 18.981813C5.672726 18.589086 5.083635 17.989086 4.407271 16.789086C3.22909 14.694541 2.792726 11.999997 2.792726 9.381816C2.792726 4.538181 4.167272 1.690909 6.076362-.24C6.109089-.272727 6.109089-.305454 6.109089-.327273C6.109089-.436364 6.032726-.436364 5.901817-.436364C5.759998-.436364 5.73818-.436364 5.639998-.349091C4.603635 .545454 3.436363 2.061818 2.683636 4.363635C2.214545 5.803635 1.963636 7.559998 1.963636 9.370906C1.963636 11.95636 2.432727 14.879996 4.145453 17.410904C4.439999 17.836359 4.843635 18.283631 4.843635 18.29454C4.952726 18.425449 5.105453 18.599995 5.192726 18.676358L5.61818 19.069086Z'/> +<path id='g0-17' d='M4.538181 9.370906C4.538181 6.239998 3.807272 2.912726 1.658181 .458182C1.505454 .283636 1.101818-.141818 .84-.370909C.763636-.436364 .741818-.436364 .6-.436364C.490909-.436364 .392727-.436364 .392727-.327273C.392727-.283636 .436364-.24 .458182-.218182C.829091 .163636 1.418181 .763636 2.094545 1.963636C3.272726 4.058181 3.70909 6.752725 3.70909 9.370906C3.70909 14.105451 2.399999 16.996359 .436364 18.992722C.414545 19.01454 .392727 19.047267 .392727 19.079995C.392727 19.189086 .490909 19.189086 .6 19.189086C.741818 19.189086 .763636 19.189086 .861818 19.101813C1.898181 18.207268 3.065454 16.690904 3.818181 14.389087C4.298181 12.894542 4.538181 11.12727 4.538181 9.370906Z'/> +<path id='g0-80' d='M4.592726 5.825453L.72 10.614542C.632727 10.723633 .621818 10.745452 .621818 10.789088C.621818 10.909088 .72 10.909088 .916363 10.909088H9.959997L10.898179 8.192725H10.625452C10.352724 9.010907 9.621816 9.676361 8.694543 9.992724C8.519998 10.04727 7.767271 10.309088 6.163635 10.309088H1.527272L5.312726 5.629089C5.389089 5.530908 5.410908 5.49818 5.410908 5.454544S5.399998 5.399998 5.334544 5.301817L1.78909 .436364H6.109089C7.352725 .436364 9.861815 .512727 10.625452 2.552727H10.898179L9.959997 0H.916363C.621818 0 .610909 .010909 .610909 .349091L4.592726 5.825453Z'/> +<path id='g2-101' d='M2.039999-2.519999C2.356363-2.519999 3.163635-2.541817 3.70909-2.770908C4.472726-3.098181 4.527271-3.741817 4.527271-3.894544C4.527271-4.374544 4.112726-4.821817 3.359999-4.821817C2.14909-4.821817 .501818-3.763635 .501818-1.854545C.501818-.741818 1.145454 .12 2.214545 .12C3.774544 .12 4.690908-1.036363 4.690908-1.167272C4.690908-1.232727 4.625453-1.309091 4.559999-1.309091C4.505453-1.309091 4.483635-1.287272 4.418181-1.2C3.556363-.12 2.367272-.12 2.236363-.12C1.385454-.12 1.287272-1.036363 1.287272-1.385454C1.287272-1.516363 1.298181-1.854545 1.461818-2.519999H2.039999ZM1.527272-2.759999C1.952727-4.418181 3.076363-4.581817 3.359999-4.581817C3.872726-4.581817 4.167272-4.265453 4.167272-3.894544C4.167272-2.759999 2.421818-2.759999 1.974545-2.759999H1.527272Z'/> +<path id='g2-112' d='M.490909 1.330909C.403636 1.701818 .381818 1.778181-.098182 1.778181C-.229091 1.778181-.349091 1.778181-.349091 1.985454C-.349091 2.072727-.294545 2.116363-.207273 2.116363C.087273 2.116363 .403636 2.083636 .709091 2.083636C1.069091 2.083636 1.44 2.116363 1.78909 2.116363C1.843636 2.116363 1.985454 2.116363 1.985454 1.898181C1.985454 1.778181 1.876363 1.778181 1.723636 1.778181C1.178181 1.778181 1.178181 1.701818 1.178181 1.603636C1.178181 1.472727 1.636363-.305454 1.712727-.578182C1.854545-.261818 2.159999 .12 2.716363 .12C3.981817 .12 5.345453-1.472727 5.345453-3.076363C5.345453-4.101817 4.723635-4.821817 3.894544-4.821817C3.34909-4.821817 2.825454-4.42909 2.465454-4.003635C2.356363-4.592726 1.887272-4.821817 1.483636-4.821817C.981818-4.821817 .774545-4.396362 .676363-4.199999C.48-3.82909 .338182-3.174545 .338182-3.141817C.338182-3.032726 .447273-3.032726 .469091-3.032726C.578182-3.032726 .589091-3.043636 .654545-3.283635C.84-4.058181 1.058182-4.581817 1.450909-4.581817C1.636363-4.581817 1.78909-4.494544 1.78909-4.079999C1.78909-3.82909 1.756363-3.70909 1.712727-3.523635L.490909 1.330909ZM2.410908-3.403635C2.487272-3.698181 2.781817-4.003635 2.978181-4.167272C3.359999-4.505453 3.676363-4.581817 3.861817-4.581817C4.298181-4.581817 4.559999-4.199999 4.559999-3.556363S4.199999-1.658181 4.003635-1.243636C3.632726-.48 3.10909-.12 2.705454-.12C1.985454-.12 1.843636-1.025454 1.843636-1.090909C1.843636-1.112727 1.843636-1.134545 1.876363-1.265454L2.410908-3.403635Z'/> +<path id='g2-120' d='M3.643635-3.294545C3.70909-3.578181 3.959999-4.581817 4.723635-4.581817C4.77818-4.581817 5.039999-4.581817 5.269089-4.439999C4.963635-4.385453 4.745453-4.112726 4.745453-3.850908C4.745453-3.676363 4.865453-3.46909 5.159999-3.46909C5.399998-3.46909 5.749089-3.665454 5.749089-4.101817C5.749089-4.66909 5.105453-4.821817 4.734544-4.821817C4.101817-4.821817 3.719999-4.243635 3.58909-3.992726C3.316363-4.712726 2.727272-4.821817 2.410908-4.821817C1.276363-4.821817 .654545-3.414545 .654545-3.141817C.654545-3.032726 .763636-3.032726 .785454-3.032726C.872727-3.032726 .905454-3.054545 .927272-3.152726C1.298181-4.30909 2.018181-4.581817 2.38909-4.581817C2.596363-4.581817 2.978181-4.483635 2.978181-3.850908C2.978181-3.512726 2.792726-2.781817 2.38909-1.254545C2.214545-.578182 1.832727-.12 1.352727-.12C1.287272-.12 1.036363-.12 .807273-.261818C1.08-.316364 1.32-.545454 1.32-.850909C1.32-1.145454 1.08-1.232727 .916363-1.232727C.589091-1.232727 .316364-.949091 .316364-.6C.316364-.098182 .861818 .12 1.341818 .12C2.061818 .12 2.454545-.643636 2.487272-.709091C2.618181-.305454 3.010908 .12 3.665454 .12C4.78909 .12 5.410908-1.287272 5.410908-1.56C5.410908-1.66909 5.312726-1.66909 5.279999-1.66909C5.181817-1.66909 5.159999-1.625454 5.13818-1.54909C4.77818-.381818 4.036363-.12 3.687272-.12C3.261817-.12 3.087272-.469091 3.087272-.84C3.087272-1.08 3.152726-1.32 3.272726-1.799999L3.643635-3.294545Z'/> +<path id='g4-66' d='M2.421818-3.992726V-6.676362C2.421818-7.036362 2.443636-7.112725 2.956363-7.112725H4.30909C5.367271-7.112725 5.749089-6.185453 5.749089-5.607271C5.749089-4.90909 5.214544-3.992726 4.003635-3.992726H2.421818ZM4.996362-3.894544C6.054544-4.101817 6.807271-4.799999 6.807271-5.607271C6.807271-6.556362 5.803635-7.450907 4.385453-7.450907H.392727V-7.112725H.654545C1.494545-7.112725 1.516363-6.992725 1.516363-6.599998V-.850909C1.516363-.458182 1.494545-.338182 .654545-.338182H.392727V0H4.66909C6.119998 0 7.101816-.970909 7.101816-1.996363C7.101816-2.945454 6.207271-3.763635 4.996362-3.894544ZM4.319999-.338182H2.956363C2.443636-.338182 2.421818-.414545 2.421818-.774545V-3.752726H4.472726C5.552726-3.752726 6.010907-2.738181 6.010907-2.007272C6.010907-1.232727 5.443635-.338182 4.319999-.338182Z'/> +<path id='g4-80' d='M2.476363-3.447272H4.319999C5.629089-3.447272 6.807271-4.330908 6.807271-5.421817C6.807271-6.490907 5.727271-7.450907 4.232726-7.450907H.381818V-7.112725H.643636C1.483636-7.112725 1.505454-6.992725 1.505454-6.599998V-.850909C1.505454-.458182 1.483636-.338182 .643636-.338182H.381818V0C.763636-.032727 1.570909-.032727 1.985454-.032727S3.218181-.032727 3.599999 0V-.338182H3.338181C2.498181-.338182 2.476363-.458182 2.476363-.850909V-3.447272ZM2.443636-3.730908V-6.676362C2.443636-7.036362 2.465454-7.112725 2.978181-7.112725H3.94909C5.683635-7.112725 5.683635-5.956362 5.683635-5.421817C5.683635-4.90909 5.683635-3.730908 3.94909-3.730908H2.443636Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -65.539987)'> +<use x='70.734745' y='70.328842' xlink:href='#g4-66'/> +<use x='78.462043' y='70.328842' xlink:href='#g4-80'/> +<use x='85.886285' y='70.328842' xlink:href='#g2-101'/> +<use x='90.965848' y='70.328842' xlink:href='#g2-120'/> +<use x='97.200702' y='70.328842' xlink:href='#g2-112'/> +<use x='104.507498' y='58.219665' xlink:href='#g0-16'/> +<use x='112.218176' y='66.033594' xlink:href='#g3-49'/> +<rect x='112.218176' y='67.383405' height='.436352' width='4.234183'/> +<use x='112.218176' y='74.090736' xlink:href='#g3-52'/> +<use x='119.466021' y='62.146969' xlink:href='#g0-80'/> +<use x='130.981204' y='65.30181' xlink:href='#g3-52'/> +<use x='130.981204' y='73.554326' xlink:href='#g1-110'/> +<use x='136.119406' y='73.554326' xlink:href='#g3-61'/> +<use x='142.705913' y='73.554326' xlink:href='#g3-49'/> +<use x='149.256376' y='70.328842' xlink:href='#g2-112'/> +<use x='154.745024' y='71.965192' xlink:href='#g1-110'/> +<use x='160.381358' y='58.219665' xlink:href='#g0-17'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/bleu-ngram.svg b/_resources/bleu-ngram.svg new file mode 100644 index 0000000..23c0fe6 --- /dev/null +++ b/_resources/bleu-ngram.svg @@ -0,0 +1,86 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='169.624603pt' height='53.373425pt' viewBox='-.237481 -.22797 169.624603 53.373425'> +<defs> +<path id='g1-50' d='M4.630635-1.809215C4.758157-1.809215 4.933499-1.809215 4.933499-1.992528S4.758157-2.175841 4.630635-2.175841H1.075965C1.179577-3.283686 2.10411-4.128518 3.315567-4.128518H4.630635C4.758157-4.128518 4.933499-4.128518 4.933499-4.311831S4.758157-4.495143 4.630635-4.495143H3.291656C1.857036-4.495143 .70137-3.379328 .70137-1.992528C.70137-.597758 1.865006 .510087 3.291656 .510087H4.630635C4.758157 .510087 4.933499 .510087 4.933499 .326775S4.758157 .143462 4.630635 .143462H3.315567C2.10411 .143462 1.179577-.70137 1.075965-1.809215H4.630635Z'/> +<path id='g4-94' d='M2.11208-5.531258L.980324-4.463263L1.139726-4.27198L2.11208-5.061021L3.084433-4.27198L3.243836-4.455293L2.11208-5.531258Z'/> +<path id='g4-110' d='M3.873474-2.414944C3.873474-3.084433 3.57061-3.514819 2.733748-3.514819C1.944707-3.514819 1.586052-2.940971 1.490411-2.749689H1.482441V-3.514819L.326775-3.427148V-3.164134C.868742-3.164134 .932503-3.108344 .932503-2.717808V-.621669C.932503-.263014 .836862-.263014 .326775-.263014V0C.669489-.02391 1.020174-.02391 1.235367-.02391C1.466501-.02391 1.801245-.02391 2.14396 0V-.263014C1.633873-.263014 1.538232-.263014 1.538232-.621669V-2.064259C1.538232-2.901121 2.175841-3.291656 2.662017-3.291656S3.267746-2.948941 3.267746-2.446824V-.621669C3.267746-.263014 3.172105-.263014 2.662017-.263014V0C3.004732-.02391 3.355417-.02391 3.57061-.02391C3.801743-.02391 4.136488-.02391 4.479203 0V-.263014C3.969116-.263014 3.873474-.263014 3.873474-.621669V-2.414944Z'/> +<path id='g0-80' d='M5.033126 6.38406L.789041 11.632379C.6934 11.75193 .681445 11.775841 .681445 11.823661C.681445 11.955168 .789041 11.955168 1.004234 11.955168H10.915068L11.943213 8.978331H11.644334C11.345455 9.874969 10.544458 10.604234 9.528269 10.950934C9.336986 11.01071 8.51208 11.297634 6.75467 11.297634H1.673724L5.822167 6.168867C5.905853 6.06127 5.929763 6.025405 5.929763 5.977584S5.917808 5.917808 5.846077 5.810212L1.960648 .478207H6.694894C8.057783 .478207 10.807472 .561893 11.644334 2.797509H11.943213L10.915068 0H1.004234C.681445 0 .669489 .011955 .669489 .382565L5.033126 6.38406Z'/> +<path id='g5-40' d='M3.88543 2.905106C3.88543 2.86924 3.88543 2.84533 3.682192 2.642092C2.486675 1.43462 1.817186-.537983 1.817186-2.976837C1.817186-5.296139 2.379078-7.292653 3.765878-8.703362C3.88543-8.810959 3.88543-8.834869 3.88543-8.870735C3.88543-8.942466 3.825654-8.966376 3.777833-8.966376C3.622416-8.966376 2.642092-8.105604 2.056289-6.933998C1.446575-5.726526 1.171606-4.447323 1.171606-2.976837C1.171606-1.912827 1.338979-.490162 1.960648 .789041C2.666002 2.223661 3.646326 3.000747 3.777833 3.000747C3.825654 3.000747 3.88543 2.976837 3.88543 2.905106Z'/> +<path id='g5-41' d='M3.371357-2.976837C3.371357-3.88543 3.251806-5.36787 2.582316-6.75467C1.876961-8.18929 .896638-8.966376 .765131-8.966376C.71731-8.966376 .657534-8.942466 .657534-8.870735C.657534-8.834869 .657534-8.810959 .860772-8.607721C2.056289-7.400249 2.725778-5.427646 2.725778-2.988792C2.725778-.669489 2.163885 1.327024 .777086 2.737733C.657534 2.84533 .657534 2.86924 .657534 2.905106C.657534 2.976837 .71731 3.000747 .765131 3.000747C.920548 3.000747 1.900872 2.139975 2.486675 .968369C3.096389-.251059 3.371357-1.542217 3.371357-2.976837Z'/> +<path id='g5-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g5-110' d='M5.32005-2.905106C5.32005-4.016936 5.32005-4.351681 5.045081-4.734247C4.698381-5.200498 4.136488-5.272229 3.730012-5.272229C2.570361-5.272229 2.116065-4.27995 2.020423-4.040847H2.008468V-5.272229L.382565-5.140722V-4.794022C1.195517-4.794022 1.291158-4.710336 1.291158-4.124533V-.884682C1.291158-.3467 1.159651-.3467 .382565-.3467V0C.6934-.02391 1.338979-.02391 1.673724-.02391C2.020423-.02391 2.666002-.02391 2.976837 0V-.3467C2.211706-.3467 2.068244-.3467 2.068244-.884682V-3.108344C2.068244-4.363636 2.893151-5.033126 3.634371-5.033126S4.542964-4.423412 4.542964-3.694147V-.884682C4.542964-.3467 4.411457-.3467 3.634371-.3467V0C3.945205-.02391 4.590785-.02391 4.925529-.02391C5.272229-.02391 5.917808-.02391 6.228643 0V-.3467C5.630884-.3467 5.332005-.3467 5.32005-.705355V-2.905106Z'/> +<path id='g2-97' d='M3.124284-3.036613C3.052553-3.172105 2.82142-3.514819 2.335243-3.514819C1.3868-3.514819 .342715-2.406974 .342715-1.227397C.342715-.398506 .876712 .079701 1.490411 .079701C2.000498 .079701 2.438854-.326775 2.582316-.486177C2.725778 .063761 3.267746 .079701 3.363387 .079701C3.730012 .079701 3.913325-.223163 3.977086-.358655C4.136488-.645579 4.24807-1.107846 4.24807-1.139726C4.24807-1.187547 4.216189-1.243337 4.120548-1.243337S4.008966-1.195517 3.961146-.996264C3.849564-.557908 3.698132-.143462 3.387298-.143462C3.203985-.143462 3.132254-.294894 3.132254-.518057C3.132254-.653549 3.203985-.924533 3.251806-1.123786S3.419178-1.801245 3.451059-1.944707L3.610461-2.550436C3.650311-2.741719 3.737983-3.076463 3.737983-3.116314C3.737983-3.299626 3.58655-3.363387 3.482939-3.363387C3.363387-3.363387 3.164134-3.283686 3.124284-3.036613ZM2.582316-.860772C2.183811-.310834 1.769365-.143462 1.514321-.143462C1.147696-.143462 .964384-.478207 .964384-.892653C.964384-1.267248 1.179577-2.12005 1.354919-2.470735C1.586052-2.956912 1.976588-3.291656 2.343213-3.291656C2.86127-3.291656 3.012702-2.709838 3.012702-2.614197C3.012702-2.582316 2.81345-1.801245 2.765629-1.594022C2.662017-1.219427 2.662017-1.203487 2.582316-.860772Z'/> +<path id='g2-99' d='M3.259776-3.052553C2.933001-3.012702 2.82939-2.765629 2.82939-2.606227C2.82939-2.375093 3.036613-2.311333 3.140224-2.311333C3.180075-2.311333 3.58655-2.343213 3.58655-2.82939S3.060523-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.541968 .868742 .079701 1.753425 .079701C3.012702 .079701 3.674222-.72528 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.769365-.143462C1.338979-.143462 .996264-.406476 .996264-1.012204C.996264-1.362889 1.155666-2.207721 1.530262-2.701868C1.880946-3.148194 2.279452-3.291656 2.590286-3.291656C2.685928-3.291656 3.052553-3.283686 3.259776-3.052553Z'/> +<path id='g2-103' d='M3.953176-2.940971C3.985056-3.060523 3.985056-3.108344 3.985056-3.116314C3.985056-3.299626 3.833624-3.363387 3.730012-3.363387C3.5467-3.363387 3.403238-3.219925 3.371357-3.052553C3.307597-3.172105 3.068493-3.514819 2.590286-3.514819C1.641843-3.514819 .605729-2.454795 .605729-1.291158C.605729-.430386 1.171606 0 1.761395 0C2.12802 0 2.454795-.199253 2.717808-.430386L2.542466 .286924C2.454795 .621669 2.399004 .852802 2.088169 1.115816C1.753425 1.40274 1.45056 1.40274 1.251308 1.40274S.860772 1.39477 .669489 1.354919C.844832 1.259278 .924533 1.091905 .924533 .956413C.924533 .765131 .781071 .661519 .613699 .661519C.414446 .661519 .167372 .820922 .167372 1.139726C.167372 1.594022 .789041 1.625903 1.267248 1.625903C2.399004 1.625903 2.964882 1.020174 3.084433 .533998L3.953176-2.940971ZM2.87721-1.052055C2.82142-.828892 2.646077-.669489 2.478705-.518057C2.406974-.462267 2.10411-.223163 1.777335-.223163C1.458531-.223163 1.227397-.486177 1.227397-.964384C1.227397-1.299128 1.42665-2.16787 1.641843-2.574346C1.896887-3.028643 2.271482-3.291656 2.590286-3.291656C3.124284-3.291656 3.267746-2.701868 3.267746-2.630137L3.243836-2.518555L2.87721-1.052055Z'/> +<path id='g2-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g2-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g2-109' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.745455-1.928767 1.793275-2.12005 1.809215-2.199751C1.825156-2.239601 2.088169-2.757659 2.438854-3.020672C2.709838-3.227895 2.972852-3.291656 3.196015-3.291656C3.490909-3.291656 3.650311-3.116314 3.650311-2.749689C3.650311-2.558406 3.602491-2.375093 3.514819-2.016438C3.459029-1.809215 3.323537-1.275218 3.275716-1.060025L3.156164-.581818C3.116314-.446326 3.060523-.207223 3.060523-.167372C3.060523 .01594 3.211955 .079701 3.315567 .079701C3.459029 .079701 3.57858-.01594 3.634371-.111582C3.658281-.159402 3.722042-.430386 3.761893-.597758L3.945205-1.307098C3.969116-1.42665 4.048817-1.729514 4.072727-1.849066C4.184309-2.279452 4.184309-2.287422 4.367621-2.550436C4.630635-2.940971 5.00523-3.291656 5.539228-3.291656C5.826152-3.291656 5.993524-3.124284 5.993524-2.749689C5.993524-2.311333 5.65878-1.39477 5.507347-1.012204C5.427646-.804981 5.403736-.749191 5.403736-.597758C5.403736-.143462 5.778331 .079701 6.121046 .079701C6.902117 .079701 7.228892-1.036115 7.228892-1.139726C7.228892-1.219427 7.165131-1.243337 7.10934-1.243337C7.013699-1.243337 6.997758-1.187547 6.973848-1.107846C6.782565-.446326 6.447821-.143462 6.144956-.143462C6.017435-.143462 5.953674-.223163 5.953674-.406476S6.017435-.765131 6.097136-.964384C6.216687-1.267248 6.567372-2.183811 6.567372-2.630137C6.567372-3.227895 6.152927-3.514819 5.579078-3.514819C5.029141-3.514819 4.574844-3.227895 4.216189-2.733748C4.152428-3.371357 3.642341-3.514819 3.227895-3.514819C2.86127-3.514819 2.375093-3.387298 1.936737-2.81345C1.880946-3.291656 1.498381-3.514819 1.123786-3.514819C.844832-3.514819 .645579-3.347447 .510087-3.076463C.318804-2.701868 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g2-110' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.833126-2.279452 1.833126-2.287422 2.016438-2.550436C2.279452-2.940971 2.654047-3.291656 3.188045-3.291656C3.474969-3.291656 3.642341-3.124284 3.642341-2.749689C3.642341-2.311333 3.307597-1.40274 3.156164-1.012204C3.052553-.749191 3.052553-.70137 3.052553-.597758C3.052553-.143462 3.427148 .079701 3.769863 .079701C4.550934 .079701 4.877709-1.036115 4.877709-1.139726C4.877709-1.219427 4.813948-1.243337 4.758157-1.243337C4.662516-1.243337 4.646575-1.187547 4.622665-1.107846C4.431382-.454296 4.096638-.143462 3.793773-.143462C3.666252-.143462 3.602491-.223163 3.602491-.406476S3.666252-.765131 3.745953-.964384C3.865504-1.267248 4.216189-2.183811 4.216189-2.630137C4.216189-3.227895 3.801743-3.514819 3.227895-3.514819C2.582316-3.514819 2.16787-3.124284 1.936737-2.82142C1.880946-3.259776 1.530262-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.318804-2.709838 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g2-112' d='M.414446 .964384C.350685 1.219427 .334745 1.283188 .01594 1.283188C-.095641 1.283188-.191283 1.283188-.191283 1.43462C-.191283 1.506351-.119552 1.546202-.079701 1.546202C0 1.546202 .03188 1.522291 .621669 1.522291C1.195517 1.522291 1.362889 1.546202 1.41868 1.546202C1.45056 1.546202 1.570112 1.546202 1.570112 1.39477C1.570112 1.283188 1.458531 1.283188 1.362889 1.283188C.980324 1.283188 .980324 1.235367 .980324 1.163636C.980324 1.107846 1.123786 .541968 1.362889-.390535C1.466501-.207223 1.713574 .079701 2.14396 .079701C3.124284 .079701 4.144458-1.052055 4.144458-2.207721C4.144458-2.996762 3.634371-3.514819 2.996762-3.514819C2.518555-3.514819 2.13599-3.188045 1.904857-2.948941C1.737484-3.514819 1.203487-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.326775-2.725778 .239103-2.319303 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.629639-2.83736 .773101-3.291656 1.099875-3.291656C1.299128-3.291656 1.354919-3.108344 1.354919-2.917061C1.354919-2.83736 1.323039-2.646077 1.307098-2.582316L.414446 .964384ZM1.880946-2.454795C1.920797-2.590286 1.920797-2.606227 2.040349-2.749689C2.343213-3.108344 2.685928-3.291656 2.972852-3.291656C3.371357-3.291656 3.52279-2.901121 3.52279-2.542466C3.52279-2.247572 3.347447-1.39477 3.108344-.924533C2.901121-.494147 2.518555-.143462 2.14396-.143462C1.601993-.143462 1.474471-.765131 1.474471-.820922C1.474471-.836862 1.490411-.924533 1.498381-.948443L1.880946-2.454795Z'/> +<path id='g2-114' d='M1.538232-1.099875C1.625903-1.44259 1.713574-1.785305 1.793275-2.13599C1.801245-2.15193 1.857036-2.383064 1.865006-2.422914C1.888917-2.494645 2.088169-2.82142 2.295392-3.020672C2.550436-3.251806 2.82142-3.291656 2.964882-3.291656C3.052553-3.291656 3.196015-3.283686 3.307597-3.188045C2.964882-3.116314 2.917061-2.82142 2.917061-2.749689C2.917061-2.574346 3.052553-2.454795 3.227895-2.454795C3.443088-2.454795 3.682192-2.630137 3.682192-2.948941C3.682192-3.235866 3.435118-3.514819 2.980822-3.514819C2.438854-3.514819 2.072229-3.156164 1.904857-2.940971C1.745455-3.514819 1.203487-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.326775-2.725778 .239103-2.319303 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.243337 .079701 1.299128-.143462 1.362889-.414446L1.538232-1.099875Z'/> +<path id='g2-121' d='M4.128518-3.004732C4.160399-3.116314 4.160399-3.132254 4.160399-3.188045C4.160399-3.387298 4.000996-3.435118 3.905355-3.435118C3.865504-3.435118 3.682192-3.427148 3.57858-3.219925C3.56264-3.180075 3.490909-2.893151 3.451059-2.725778L2.972852-.812951C2.964882-.789041 2.622167-.143462 2.040349-.143462C1.649813-.143462 1.514321-.430386 1.514321-.789041C1.514321-1.251308 1.785305-1.960648 1.968618-2.422914C2.048319-2.622167 2.072229-2.693898 2.072229-2.83736C2.072229-3.275716 1.721544-3.514819 1.354919-3.514819C.565878-3.514819 .239103-2.391034 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.70137-3.012702 1.044085-3.291656 1.331009-3.291656C1.45056-3.291656 1.522291-3.211955 1.522291-3.028643C1.522291-2.86127 1.458531-2.677958 1.40274-2.534496C1.075965-1.689664 .940473-1.283188 .940473-.908593C.940473-.127522 1.530262 .079701 2.000498 .079701C2.375093 .079701 2.646077-.087671 2.83736-.270984C2.725778 .175342 2.646077 .486177 2.343213 .868742C2.080199 1.195517 1.761395 1.40274 1.40274 1.40274C1.267248 1.40274 .964384 1.378829 .804981 1.139726C1.227397 1.107846 1.259278 .749191 1.259278 .70137C1.259278 .510087 1.115816 .406476 .948443 .406476C.773101 .406476 .494147 .541968 .494147 .932503C.494147 1.307098 .836862 1.625903 1.40274 1.625903C2.215691 1.625903 3.132254 .972354 3.371357 .00797L4.128518-3.004732Z'/> +<path id='g3-97' d='M3.598506-1.422665C3.53873-1.219427 3.53873-1.195517 3.371357-.968369C3.108344-.633624 2.582316-.119552 2.020423-.119552C1.530262-.119552 1.255293-.561893 1.255293-1.267248C1.255293-1.924782 1.625903-3.263761 1.853051-3.765878C2.259527-4.60274 2.82142-5.033126 3.287671-5.033126C4.076712-5.033126 4.23213-4.052802 4.23213-3.957161C4.23213-3.945205 4.196264-3.789788 4.184309-3.765878L3.598506-1.422665ZM4.363636-4.483188C4.23213-4.794022 3.90934-5.272229 3.287671-5.272229C1.936737-5.272229 .478207-3.526775 .478207-1.75741C.478207-.573848 1.171606 .119552 1.984558 .119552C2.642092 .119552 3.203985-.394521 3.53873-.789041C3.658281-.083686 4.220174 .119552 4.578829 .119552S5.224408-.095641 5.439601-.526027C5.630884-.932503 5.798257-1.661768 5.798257-1.709589C5.798257-1.769365 5.750436-1.817186 5.678705-1.817186C5.571108-1.817186 5.559153-1.75741 5.511333-1.578082C5.332005-.872727 5.104857-.119552 4.614695-.119552C4.267995-.119552 4.244085-.430386 4.244085-.669489C4.244085-.944458 4.27995-1.075965 4.387547-1.542217C4.471233-1.841096 4.531009-2.10411 4.62665-2.450809C5.068991-4.244085 5.176588-4.674471 5.176588-4.746202C5.176588-4.913574 5.045081-5.045081 4.865753-5.045081C4.483188-5.045081 4.387547-4.62665 4.363636-4.483188Z'/> +<path id='g3-99' d='M4.674471-4.495143C4.447323-4.495143 4.339726-4.495143 4.172354-4.351681C4.100623-4.291905 3.969116-4.112578 3.969116-3.921295C3.969116-3.682192 4.148443-3.53873 4.375592-3.53873C4.662516-3.53873 4.985305-3.777833 4.985305-4.25604C4.985305-4.829888 4.435367-5.272229 3.610461-5.272229C2.044334-5.272229 .478207-3.56264 .478207-1.865006C.478207-.824907 1.123786 .119552 2.343213 .119552C3.969116 .119552 4.99726-1.147696 4.99726-1.303113C4.99726-1.374844 4.925529-1.43462 4.877709-1.43462C4.841843-1.43462 4.829888-1.422665 4.722291-1.315068C3.957161-.298879 2.82142-.119552 2.367123-.119552C1.542217-.119552 1.279203-.836862 1.279203-1.43462C1.279203-1.853051 1.482441-3.012702 1.912827-3.825654C2.223661-4.387547 2.86924-5.033126 3.622416-5.033126C3.777833-5.033126 4.435367-5.009215 4.674471-4.495143Z'/> +<path id='g3-103' d='M4.040847-1.518306C3.993026-1.327024 3.969116-1.279203 3.813699-1.099875C3.323537-.466252 2.82142-.239103 2.450809-.239103C2.056289-.239103 1.685679-.549938 1.685679-1.374844C1.685679-2.008468 2.044334-3.347447 2.307347-3.88543C2.654047-4.554919 3.19203-5.033126 3.694147-5.033126C4.483188-5.033126 4.638605-4.052802 4.638605-3.981071L4.60274-3.813699L4.040847-1.518306ZM4.782067-4.483188C4.62665-4.829888 4.291905-5.272229 3.694147-5.272229C2.391034-5.272229 .908593-3.634371 .908593-1.853051C.908593-.609714 1.661768 0 2.426899 0C3.060523 0 3.622416-.502117 3.837609-.74122L3.574595 .334745C3.407223 .992279 3.335492 1.291158 2.905106 1.709589C2.414944 2.199751 1.960648 2.199751 1.697634 2.199751C1.338979 2.199751 1.0401 2.175841 .74122 2.080199C1.123786 1.972603 1.219427 1.637858 1.219427 1.506351C1.219427 1.315068 1.075965 1.123786 .812951 1.123786C.526027 1.123786 .215193 1.362889 .215193 1.75741C.215193 2.247572 .705355 2.438854 1.721544 2.438854C3.263761 2.438854 4.064757 1.446575 4.220174 .800996L5.547198-4.554919C5.583064-4.698381 5.583064-4.722291 5.583064-4.746202C5.583064-4.913574 5.451557-5.045081 5.272229-5.045081C4.985305-5.045081 4.817933-4.805978 4.782067-4.483188Z'/> +<path id='g3-109' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-4.124533 5.057036-3.88543 4.913574-3.323537L4.614695-2.092154C4.519054-1.733499 4.291905-.848817 4.267995-.71731C4.220174-.537983 4.148443-.227148 4.148443-.179328C4.148443-.011955 4.27995 .119552 4.459278 .119552C4.817933 .119552 4.877709-.155417 4.985305-.585803L5.702615-3.443088C5.726526-3.53873 6.348194-5.033126 7.663263-5.033126C8.141469-5.033126 8.308842-4.674471 8.308842-4.196264C8.308842-3.526775 7.84259-2.223661 7.579577-1.506351C7.47198-1.219427 7.412204-1.06401 7.412204-.848817C7.412204-.310834 7.782814 .119552 8.356663 .119552C9.468493 .119552 9.886924-1.637858 9.886924-1.709589C9.886924-1.769365 9.839103-1.817186 9.767372-1.817186C9.659776-1.817186 9.647821-1.78132 9.588045-1.578082C9.313076-.621669 8.870735-.119552 8.392528-.119552C8.272976-.119552 8.081694-.131507 8.081694-.514072C8.081694-.824907 8.225156-1.207472 8.272976-1.338979C8.488169-1.912827 9.026152-3.323537 9.026152-4.016936C9.026152-4.734247 8.607721-5.272229 7.699128-5.272229C6.898132-5.272229 6.252553-4.817933 5.774346-4.112578C5.738481-4.758157 5.34396-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.207472-5.272229 .932503-5.104857 .705355-4.65056C.490162-4.220174 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.812951-4.327771 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +<path id='g3-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +<path id='g3-111' d='M5.451557-3.287671C5.451557-4.423412 4.710336-5.272229 3.622416-5.272229C2.044334-5.272229 .490162-3.550685 .490162-1.865006C.490162-.729265 1.231382 .119552 2.319303 .119552C3.90934 .119552 5.451557-1.601993 5.451557-3.287671ZM2.331258-.119552C1.733499-.119552 1.291158-.597758 1.291158-1.43462C1.291158-1.984558 1.578082-3.203985 1.912827-3.801743C2.450809-4.722291 3.120299-5.033126 3.610461-5.033126C4.196264-5.033126 4.65056-4.554919 4.65056-3.718057C4.65056-3.239851 4.399502-1.960648 3.945205-1.231382C3.455044-.430386 2.797509-.119552 2.331258-.119552Z'/> +<path id='g3-112' d='M.514072 1.518306C.430386 1.876961 .382565 1.972603-.107597 1.972603C-.251059 1.972603-.37061 1.972603-.37061 2.199751C-.37061 2.223661-.358655 2.319303-.227148 2.319303C-.071731 2.319303 .095641 2.295392 .251059 2.295392H.765131C1.016189 2.295392 1.625903 2.319303 1.876961 2.319303C1.948692 2.319303 2.092154 2.319303 2.092154 2.10411C2.092154 1.972603 2.008468 1.972603 1.80523 1.972603C1.255293 1.972603 1.219427 1.888917 1.219427 1.793275C1.219427 1.649813 1.75741-.406476 1.829141-.681445C1.960648-.3467 2.283437 .119552 2.905106 .119552C4.25604 .119552 5.71457-1.637858 5.71457-3.395268C5.71457-4.495143 5.092902-5.272229 4.196264-5.272229C3.431133-5.272229 2.785554-4.531009 2.654047-4.363636C2.558406-4.961395 2.092154-5.272229 1.613948-5.272229C1.267248-5.272229 .992279-5.104857 .765131-4.65056C.549938-4.220174 .382565-3.490909 .382565-3.443088S.430386-3.335492 .514072-3.335492C.609714-3.335492 .621669-3.347447 .6934-3.622416C.872727-4.327771 1.099875-5.033126 1.578082-5.033126C1.853051-5.033126 1.948692-4.841843 1.948692-4.483188C1.948692-4.196264 1.912827-4.076712 1.865006-3.861519L.514072 1.518306ZM2.582316-3.730012C2.666002-4.064757 3.000747-4.411457 3.19203-4.578829C3.323537-4.698381 3.718057-5.033126 4.172354-5.033126C4.698381-5.033126 4.937484-4.507098 4.937484-3.88543C4.937484-3.311582 4.60274-1.960648 4.303861-1.338979C4.004981-.6934 3.455044-.119552 2.905106-.119552C2.092154-.119552 1.960648-1.147696 1.960648-1.195517C1.960648-1.231382 1.984558-1.327024 1.996513-1.3868L2.582316-3.730012Z'/> +<path id='g3-114' d='M4.65056-4.889664C4.27995-4.817933 4.088667-4.554919 4.088667-4.291905C4.088667-4.004981 4.315816-3.90934 4.483188-3.90934C4.817933-3.90934 5.092902-4.196264 5.092902-4.554919C5.092902-4.937484 4.722291-5.272229 4.124533-5.272229C3.646326-5.272229 3.096389-5.057036 2.594271-4.327771C2.510585-4.961395 2.032379-5.272229 1.554172-5.272229C1.08792-5.272229 .848817-4.913574 .705355-4.65056C.502117-4.220174 .32279-3.502864 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.812951-4.339726 1.0401-5.033126 1.518306-5.033126C1.80523-5.033126 1.888917-4.829888 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.338979 .119552 1.566127 .035866 1.637858-.203238C1.673724-.298879 2.116065-2.10411 2.187796-2.379078C2.247572-2.642092 2.319303-2.893151 2.379078-3.156164C2.426899-3.323537 2.47472-3.514819 2.510585-3.670237C2.546451-3.777833 2.86924-4.363636 3.16812-4.62665C3.311582-4.758157 3.622416-5.033126 4.112578-5.033126C4.303861-5.033126 4.495143-4.99726 4.65056-4.889664Z'/> +<path id='g3-116' d='M2.402989-4.805978H3.502864C3.730012-4.805978 3.849564-4.805978 3.849564-5.021171C3.849564-5.152677 3.777833-5.152677 3.53873-5.152677H2.486675L2.929016-6.898132C2.976837-7.065504 2.976837-7.089415 2.976837-7.173101C2.976837-7.364384 2.82142-7.47198 2.666002-7.47198C2.570361-7.47198 2.295392-7.436115 2.199751-7.053549L1.733499-5.152677H.609714C.37061-5.152677 .263014-5.152677 .263014-4.925529C.263014-4.805978 .3467-4.805978 .573848-4.805978H1.637858L.848817-1.649813C.753176-1.231382 .71731-1.111831 .71731-.956413C.71731-.394521 1.111831 .119552 1.78132 .119552C2.988792 .119552 3.634371-1.625903 3.634371-1.709589C3.634371-1.78132 3.58655-1.817186 3.514819-1.817186C3.490909-1.817186 3.443088-1.817186 3.419178-1.769365C3.407223-1.75741 3.395268-1.745455 3.311582-1.554172C3.060523-.956413 2.510585-.119552 1.817186-.119552C1.458531-.119552 1.43462-.418431 1.43462-.681445C1.43462-.6934 1.43462-.920548 1.470486-1.06401L2.402989-4.805978Z'/> +<path id='g3-117' d='M4.076712-.6934C4.23213-.02391 4.805978 .119552 5.092902 .119552C5.475467 .119552 5.762391-.131507 5.953674-.537983C6.156912-.968369 6.312329-1.673724 6.312329-1.709589C6.312329-1.769365 6.264508-1.817186 6.192777-1.817186C6.085181-1.817186 6.073225-1.75741 6.025405-1.578082C5.810212-.753176 5.595019-.119552 5.116812-.119552C4.758157-.119552 4.758157-.514072 4.758157-.669489C4.758157-.944458 4.794022-1.06401 4.913574-1.566127C4.99726-1.888917 5.080946-2.211706 5.152677-2.546451L5.642839-4.495143C5.726526-4.794022 5.726526-4.817933 5.726526-4.853798C5.726526-5.033126 5.583064-5.152677 5.403736-5.152677C5.057036-5.152677 4.97335-4.853798 4.901619-4.554919C4.782067-4.088667 4.136488-1.518306 4.052802-1.099875C4.040847-1.099875 3.574595-.119552 2.701868-.119552C2.080199-.119552 1.960648-.657534 1.960648-1.099875C1.960648-1.78132 2.295392-2.737733 2.606227-3.53873C2.749689-3.921295 2.809465-4.076712 2.809465-4.315816C2.809465-4.829888 2.438854-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.578829 1.374844-5.033126 1.829141-5.033126C1.948692-5.033126 2.139975-5.021171 2.139975-4.638605C2.139975-4.327771 2.008468-3.981071 1.829141-3.526775C1.303113-2.10411 1.243337-1.649813 1.243337-1.291158C1.243337-.071731 2.163885 .119552 2.654047 .119552C3.419178 .119552 3.837609-.406476 4.076712-.6934Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.565683 -61.02)'> +<use x='56.413267' y='80.688613' xlink:href='#g3-112'/> +<use x='62.28841' y='82.481876' xlink:href='#g2-110'/> +<use x='71.245574' y='80.688613' xlink:href='#g5-61'/> +<use x='96.148328' y='53.798257' xlink:href='#g0-80'/> +<use x='84.866569' y='73.280906' xlink:href='#g4-110'/> +<use x='89.571216' y='73.280906' xlink:href='#g2-103'/> +<use x='93.887033' y='73.280906' xlink:href='#g2-114'/> +<use x='97.943628' y='73.280906' xlink:href='#g2-97'/> +<use x='102.441638' y='73.280906' xlink:href='#g2-109'/> +<use x='109.932164' y='73.280906' xlink:href='#g1-50'/> +<use x='116.166978' y='73.280906' xlink:href='#g4-94'/> +<use x='115.577741' y='73.280906' xlink:href='#g2-121'/> +<use x='120.049452' y='62.764709' xlink:href='#g3-99'/> +<use x='125.087441' y='62.764709' xlink:href='#g3-111'/> +<use x='130.714878' y='62.764709' xlink:href='#g3-117'/> +<use x='137.377318' y='62.764709' xlink:href='#g3-110'/> +<use x='144.364923' y='62.764709' xlink:href='#g3-116'/> +<use x='148.592083' y='64.557972' xlink:href='#g2-99'/> +<use x='152.259863' y='64.557972' xlink:href='#g2-108'/> +<use x='154.881988' y='64.557972' xlink:href='#g2-105'/> +<use x='157.765127' y='64.557972' xlink:href='#g2-112'/> +<use x='162.526037' y='62.764709' xlink:href='#g5-40'/> +<use x='167.078363' y='62.764709' xlink:href='#g5-110'/> +<use x='173.581685' y='62.764709' xlink:href='#g3-103'/> +<use x='179.615941' y='62.764709' xlink:href='#g3-114'/> +<use x='185.216415' y='62.764709' xlink:href='#g3-97'/> +<use x='191.361359' y='62.764709' xlink:href='#g3-109'/> +<use x='201.600626' y='62.764709' xlink:href='#g5-41'/> +<rect x='84.866569' y='77.460728' height='.478187' width='121.286357'/> +<use x='103.115305' y='79.922824' xlink:href='#g0-80'/> +<use x='91.833545' y='99.405473' xlink:href='#g4-110'/> +<use x='96.538193' y='99.405473' xlink:href='#g2-103'/> +<use x='100.85401' y='99.405473' xlink:href='#g2-114'/> +<use x='104.910605' y='99.405473' xlink:href='#g2-97'/> +<use x='109.408615' y='99.405473' xlink:href='#g2-109'/> +<use x='116.899141' y='99.405473' xlink:href='#g1-50'/> +<use x='123.133955' y='99.405473' xlink:href='#g4-94'/> +<use x='122.544718' y='99.405473' xlink:href='#g2-121'/> +<use x='127.016429' y='88.889276' xlink:href='#g3-99'/> +<use x='132.054418' y='88.889276' xlink:href='#g3-111'/> +<use x='137.681855' y='88.889276' xlink:href='#g3-117'/> +<use x='144.344295' y='88.889276' xlink:href='#g3-110'/> +<use x='151.3319' y='88.889276' xlink:href='#g3-116'/> +<use x='155.55906' y='88.889276' xlink:href='#g5-40'/> +<use x='160.111386' y='88.889276' xlink:href='#g5-110'/> +<use x='166.614708' y='88.889276' xlink:href='#g3-103'/> +<use x='172.648965' y='88.889276' xlink:href='#g3-114'/> +<use x='178.249438' y='88.889276' xlink:href='#g3-97'/> +<use x='184.394382' y='88.889276' xlink:href='#g3-109'/> +<use x='194.633649' y='88.889276' xlink:href='#g5-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/bleu-score-drop.png b/_resources/bleu-score-drop.png new file mode 100644 index 0000000..4fdd0ac Binary files /dev/null and b/_resources/bleu-score-drop.png differ diff --git a/_resources/c1e5b885c1aead4ff5700b0a10cc0302.png b/_resources/c1e5b885c1aead4ff5700b0a10cc0302.png new file mode 100644 index 0000000..ccc506e Binary files /dev/null and b/_resources/c1e5b885c1aead4ff5700b0a10cc0302.png differ diff --git a/_resources/c2a9b03791fddf1aabea180f18076f55.png b/_resources/c2a9b03791fddf1aabea180f18076f55.png new file mode 100644 index 0000000..68e33bc Binary files /dev/null and b/_resources/c2a9b03791fddf1aabea180f18076f55.png differ diff --git a/_resources/c2e4a299ff489309dab8e6e81864d82b.png b/_resources/c2e4a299ff489309dab8e6e81864d82b.png new file mode 100644 index 0000000..5b1d20c Binary files /dev/null and b/_resources/c2e4a299ff489309dab8e6e81864d82b.png differ diff --git a/_resources/c30291ea3fd814cde29bbc5e2f275c3c.png b/_resources/c30291ea3fd814cde29bbc5e2f275c3c.png new file mode 100644 index 0000000..084bd60 Binary files /dev/null and b/_resources/c30291ea3fd814cde29bbc5e2f275c3c.png differ diff --git a/_resources/c34ed20473a52dda3d546240f3b6e52c.png b/_resources/c34ed20473a52dda3d546240f3b6e52c.png new file mode 100644 index 0000000..d64e697 Binary files /dev/null and b/_resources/c34ed20473a52dda3d546240f3b6e52c.png differ diff --git a/_resources/c3f2d93dc2ae40260e94b36083fc20a9.png b/_resources/c3f2d93dc2ae40260e94b36083fc20a9.png new file mode 100644 index 0000000..84461f0 Binary files /dev/null and b/_resources/c3f2d93dc2ae40260e94b36083fc20a9.png differ diff --git a/_resources/c45b3dcdac107122793b14d8bdd76a0f.png b/_resources/c45b3dcdac107122793b14d8bdd76a0f.png new file mode 100644 index 0000000..ca60292 Binary files /dev/null and b/_resources/c45b3dcdac107122793b14d8bdd76a0f.png differ diff --git a/_resources/c53ad46a0de9b5989d9c61f607aad517.png b/_resources/c53ad46a0de9b5989d9c61f607aad517.png new file mode 100644 index 0000000..941fc03 Binary files /dev/null and b/_resources/c53ad46a0de9b5989d9c61f607aad517.png differ diff --git a/_resources/c7b0d8e8356eaef330da4a5ba9040df3.png b/_resources/c7b0d8e8356eaef330da4a5ba9040df3.png new file mode 100644 index 0000000..923fd71 Binary files /dev/null and b/_resources/c7b0d8e8356eaef330da4a5ba9040df3.png differ diff --git a/_resources/c95aa196527b6a81b5ad0e3e6eb4d20d.png b/_resources/c95aa196527b6a81b5ad0e3e6eb4d20d.png new file mode 100644 index 0000000..2b22d9b Binary files /dev/null and b/_resources/c95aa196527b6a81b5ad0e3e6eb4d20d.png differ diff --git a/_resources/c9a1ce3aec2e82148c02a5d70d096109.png b/_resources/c9a1ce3aec2e82148c02a5d70d096109.png new file mode 100644 index 0000000..0ad8f53 Binary files /dev/null and b/_resources/c9a1ce3aec2e82148c02a5d70d096109.png differ diff --git a/_resources/ca233fcb9a340843cc102e395c3b0a38.png b/_resources/ca233fcb9a340843cc102e395c3b0a38.png new file mode 100644 index 0000000..5fb53a0 Binary files /dev/null and b/_resources/ca233fcb9a340843cc102e395c3b0a38.png differ diff --git a/_resources/cbffc74d82c2acc542253f2a050fa649.png b/_resources/cbffc74d82c2acc542253f2a050fa649.png new file mode 100644 index 0000000..6e359a0 Binary files /dev/null and b/_resources/cbffc74d82c2acc542253f2a050fa649.png differ diff --git a/_resources/ce13e5abfb93d0f151b38e19a04cf13b.png b/_resources/ce13e5abfb93d0f151b38e19a04cf13b.png new file mode 100644 index 0000000..d480bf1 Binary files /dev/null and b/_resources/ce13e5abfb93d0f151b38e19a04cf13b.png differ diff --git a/_resources/cf910f508994db7a3d68732db2c383fb.png b/_resources/cf910f508994db7a3d68732db2c383fb.png new file mode 100644 index 0000000..7150377 Binary files /dev/null and b/_resources/cf910f508994db7a3d68732db2c383fb.png differ diff --git a/_resources/colo.png b/_resources/colo.png new file mode 100644 index 0000000..057b8f7 Binary files /dev/null and b/_resources/colo.png differ diff --git a/_resources/conv-1x1.svg b/_resources/conv-1x1.svg new file mode 100644 index 0000000..2bf203f --- /dev/null +++ b/_resources/conv-1x1.svg @@ -0,0 +1,75 @@ +<?xml version="1.0" encoding="UTF-8"?> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="338pt" height="126pt" viewBox="0 0 338 126" version="1.1"> +<g id="surface1"> +<path style="fill-rule:nonzero;fill:rgb(40%,74.901961%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 63 108 L 81 108 L 81 126 L 63 126 Z M 63 108 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 63 72 L 81 72 L 81 90 L 63 90 Z M 63 72 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 63 90 L 81 90 L 81 108 L 63 108 Z M 63 90 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 27 108 L 45 108 L 45 126 L 27 126 Z M 27 108 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 45 108 L 63 108 L 63 126 L 45 126 Z M 45 108 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(69.803922%,85.098039%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 27 72 L 45 72 L 45 90 L 27 90 Z M 27 72 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 27 90 L 45 90 L 45 108 L 27 108 Z M 27 90 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 45 72 L 63 72 L 63 90 L 45 90 Z M 45 72 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 45 90 L 63 90 L 63 108 L 45 108 Z M 45 90 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(40%,74.901961%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 54 117 L 72 117 L 72 135 L 54 135 Z M 54 117 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 54 81 L 72 81 L 72 99 L 54 99 Z M 54 81 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 54 99 L 72 99 L 72 117 L 54 117 Z M 54 99 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18 117 L 36 117 L 36 135 L 18 135 Z M 18 117 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 36 117 L 54 117 L 54 135 L 36 135 Z M 36 117 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(69.803922%,85.098039%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18 81 L 36 81 L 36 99 L 18 99 Z M 18 81 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18 99 L 36 99 L 36 117 L 18 117 Z M 18 99 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 36 81 L 54 81 L 54 99 L 36 99 Z M 36 81 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 36 99 L 54 99 L 54 117 L 36 117 Z M 36 99 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 280.628906 12.324219 C 280.628906 10.996094 280.988281 9.957031 281.703125 9.199219 C 282.417969 8.433594 283.34375 8.046875 284.476562 8.046875 C 285.21875 8.046875 285.890625 8.234375 286.488281 8.59375 C 287.085938 8.945312 287.542969 9.445312 287.855469 10.078125 C 288.167969 10.71875 288.324219 11.4375 288.324219 12.226562 C 288.324219 13.046875 288.160156 13.785156 287.835938 14.433594 C 287.511719 15.074219 287.042969 15.5625 286.429688 15.898438 C 285.832031 16.226562 285.179688 16.386719 284.476562 16.386719 C 283.722656 16.386719 283.042969 16.207031 282.445312 15.839844 C 281.847656 15.476562 281.390625 14.976562 281.078125 14.335938 C 280.777344 13.703125 280.628906 13.027344 280.628906 12.324219 Z M 281.722656 12.34375 C 281.722656 13.320312 281.984375 14.09375 282.503906 14.648438 C 283.023438 15.195312 283.683594 15.46875 284.476562 15.46875 C 285.269531 15.46875 285.929688 15.191406 286.449219 14.628906 C 286.96875 14.070312 287.230469 13.273438 287.230469 12.226562 C 287.230469 11.578125 287.121094 11.011719 286.898438 10.527344 C 286.675781 10.035156 286.351562 9.648438 285.921875 9.375 C 285.492188 9.101562 285.015625 8.964844 284.496094 8.964844 C 283.742188 8.964844 283.089844 9.226562 282.542969 9.746094 C 281.996094 10.269531 281.722656 11.132812 281.722656 12.34375 Z M 281.722656 12.34375 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 293.40625 16.25 L 293.40625 15.390625 C 292.945312 16.054688 292.332031 16.386719 291.550781 16.386719 C 291.207031 16.386719 290.890625 16.316406 290.59375 16.191406 C 290.289062 16.050781 290.066406 15.878906 289.910156 15.683594 C 289.761719 15.488281 289.664062 15.25 289.617188 14.960938 C 289.578125 14.765625 289.558594 14.453125 289.558594 14.023438 L 289.558594 10.410156 L 290.535156 10.410156 L 290.535156 13.652344 C 290.535156 14.175781 290.558594 14.523438 290.613281 14.6875 C 290.660156 14.953125 290.789062 15.15625 290.984375 15.3125 C 291.1875 15.460938 291.441406 15.527344 291.746094 15.527344 C 292.042969 15.527344 292.320312 15.460938 292.585938 15.3125 C 292.84375 15.15625 293.023438 14.953125 293.132812 14.6875 C 293.234375 14.429688 293.289062 14.042969 293.289062 13.535156 L 293.289062 10.410156 L 294.285156 10.410156 L 294.285156 16.25 Z M 293.40625 16.25 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 297.996094 15.371094 L 298.132812 16.230469 C 297.84375 16.292969 297.597656 16.328125 297.390625 16.328125 C 297.023438 16.328125 296.746094 16.269531 296.550781 16.152344 C 296.355469 16.035156 296.210938 15.890625 296.121094 15.703125 C 296.042969 15.523438 296.003906 15.136719 296.003906 14.550781 L 296.003906 11.191406 L 295.28125 11.191406 L 295.28125 10.410156 L 296.003906 10.410156 L 296.003906 8.964844 L 297 8.378906 L 297 10.410156 L 297.996094 10.410156 L 297.996094 11.191406 L 297 11.191406 L 297 14.589844 C 297 14.878906 297.011719 15.058594 297.039062 15.136719 C 297.078125 15.214844 297.136719 15.28125 297.214844 15.332031 C 297.292969 15.386719 297.402344 15.410156 297.546875 15.410156 C 297.664062 15.410156 297.8125 15.398438 297.996094 15.371094 Z M 297.996094 15.371094 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 298.953125 18.476562 L 298.953125 10.410156 L 299.851562 10.410156 L 299.851562 11.171875 C 300.058594 10.875 300.292969 10.65625 300.554688 10.507812 C 300.828125 10.367188 301.160156 10.292969 301.550781 10.292969 C 302.046875 10.292969 302.480469 10.425781 302.859375 10.683594 C 303.238281 10.933594 303.523438 11.289062 303.71875 11.757812 C 303.914062 12.226562 304.011719 12.734375 304.011719 13.28125 C 304.011719 13.882812 303.902344 14.421875 303.679688 14.902344 C 303.472656 15.386719 303.164062 15.757812 302.761719 16.015625 C 302.359375 16.257812 301.929688 16.386719 301.472656 16.386719 C 301.148438 16.386719 300.855469 16.3125 300.59375 16.171875 C 300.332031 16.03125 300.117188 15.855469 299.949219 15.644531 L 299.949219 18.476562 Z M 299.851562 13.359375 C 299.851562 14.117188 300 14.679688 300.300781 15.039062 C 300.613281 15.390625 300.984375 15.566406 301.414062 15.566406 C 301.84375 15.566406 302.214844 15.386719 302.527344 15.019531 C 302.851562 14.644531 303.015625 14.0625 303.015625 13.28125 C 303.015625 12.539062 302.859375 11.988281 302.546875 11.621094 C 302.246094 11.246094 301.882812 11.054688 301.453125 11.054688 C 301.035156 11.054688 300.664062 11.257812 300.339844 11.660156 C 300.015625 12.050781 299.851562 12.617188 299.851562 13.359375 Z M 299.851562 13.359375 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 309.035156 16.25 L 309.035156 15.390625 C 308.574219 16.054688 307.960938 16.386719 307.179688 16.386719 C 306.835938 16.386719 306.519531 16.316406 306.222656 16.191406 C 305.917969 16.050781 305.695312 15.878906 305.539062 15.683594 C 305.390625 15.488281 305.292969 15.25 305.246094 14.960938 C 305.207031 14.765625 305.1875 14.453125 305.1875 14.023438 L 305.1875 10.410156 L 306.164062 10.410156 L 306.164062 13.652344 C 306.164062 14.175781 306.1875 14.523438 306.242188 14.6875 C 306.289062 14.953125 306.417969 15.15625 306.613281 15.3125 C 306.816406 15.460938 307.070312 15.527344 307.375 15.527344 C 307.671875 15.527344 307.949219 15.460938 308.214844 15.3125 C 308.472656 15.15625 308.652344 14.953125 308.761719 14.6875 C 308.863281 14.429688 308.917969 14.042969 308.917969 13.535156 L 308.917969 10.410156 L 309.914062 10.410156 L 309.914062 16.25 Z M 309.035156 16.25 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 313.628906 15.371094 L 313.765625 16.230469 C 313.476562 16.292969 313.226562 16.328125 313.023438 16.328125 C 312.65625 16.328125 312.378906 16.269531 312.183594 16.152344 C 311.988281 16.035156 311.839844 15.890625 311.753906 15.703125 C 311.675781 15.523438 311.636719 15.136719 311.636719 14.550781 L 311.636719 11.191406 L 310.914062 11.191406 L 310.914062 10.410156 L 311.636719 10.410156 L 311.636719 8.964844 L 312.632812 8.378906 L 312.632812 10.410156 L 313.628906 10.410156 L 313.628906 11.191406 L 312.632812 11.191406 L 312.632812 14.589844 C 312.632812 14.878906 312.640625 15.058594 312.671875 15.136719 C 312.710938 15.214844 312.769531 15.28125 312.847656 15.332031 C 312.925781 15.386719 313.03125 15.410156 313.179688 15.410156 C 313.296875 15.410156 313.441406 15.398438 313.628906 15.371094 Z M 313.628906 15.371094 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 151.398438 16.25 L 151.398438 8.203125 L 152.472656 8.203125 L 152.472656 12.1875 L 156.476562 8.203125 L 157.902344 8.203125 L 154.542969 11.464844 L 158.058594 16.25 L 156.652344 16.25 L 153.78125 12.167969 L 152.472656 13.457031 L 152.472656 16.25 Z M 151.398438 16.25 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 162.808594 14.375 L 163.84375 14.492188 C 163.671875 15.09375 163.363281 15.5625 162.925781 15.898438 C 162.496094 16.226562 161.9375 16.386719 161.265625 16.386719 C 160.414062 16.386719 159.742188 16.128906 159.234375 15.605469 C 158.734375 15.074219 158.492188 14.332031 158.492188 13.378906 C 158.492188 12.402344 158.746094 11.648438 159.253906 11.113281 C 159.761719 10.566406 160.414062 10.292969 161.226562 10.292969 C 162.007812 10.292969 162.636719 10.5625 163.121094 11.09375 C 163.613281 11.617188 163.863281 12.359375 163.863281 13.320312 C 163.863281 13.390625 163.863281 13.476562 163.863281 13.59375 L 159.507812 13.59375 C 159.546875 14.234375 159.726562 14.722656 160.054688 15.058594 C 160.375 15.398438 160.785156 15.566406 161.285156 15.566406 C 161.644531 15.566406 161.953125 15.476562 162.203125 15.292969 C 162.460938 15.097656 162.660156 14.796875 162.808594 14.375 Z M 159.566406 12.773438 L 162.828125 12.773438 C 162.789062 12.28125 162.660156 11.910156 162.457031 11.660156 C 162.144531 11.285156 161.734375 11.09375 161.226562 11.09375 C 160.765625 11.09375 160.386719 11.25 160.074219 11.5625 C 159.769531 11.867188 159.605469 12.265625 159.566406 12.773438 Z M 159.566406 12.773438 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 165.0625 16.25 L 165.0625 10.410156 L 165.960938 10.410156 L 165.960938 11.308594 C 166.179688 10.894531 166.390625 10.621094 166.585938 10.488281 C 166.78125 10.359375 166.996094 10.292969 167.230469 10.292969 C 167.550781 10.292969 167.894531 10.398438 168.246094 10.605469 L 167.894531 11.523438 C 167.660156 11.382812 167.414062 11.308594 167.171875 11.308594 C 166.960938 11.308594 166.765625 11.375 166.585938 11.503906 C 166.414062 11.636719 166.292969 11.816406 166.214844 12.050781 C 166.105469 12.402344 166.058594 12.789062 166.058594 13.203125 L 166.058594 16.25 Z M 165.0625 16.25 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 168.828125 16.25 L 168.828125 10.410156 L 169.726562 10.410156 L 169.726562 11.25 C 170.140625 10.617188 170.761719 10.292969 171.582031 10.292969 C 171.933594 10.292969 172.25 10.359375 172.539062 10.488281 C 172.835938 10.605469 173.054688 10.773438 173.203125 10.976562 C 173.34375 11.171875 173.445312 11.414062 173.515625 11.699219 C 173.554688 11.882812 173.574219 12.203125 173.574219 12.65625 L 173.574219 16.25 L 172.578125 16.25 L 172.578125 12.695312 C 172.578125 12.296875 172.539062 11.992188 172.460938 11.796875 C 172.382812 11.601562 172.246094 11.445312 172.050781 11.328125 C 171.855469 11.203125 171.625 11.132812 171.367188 11.132812 C 170.945312 11.132812 170.585938 11.269531 170.273438 11.542969 C 169.96875 11.804688 169.824219 12.3125 169.824219 13.066406 L 169.824219 16.25 Z M 168.828125 16.25 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 179.070312 14.375 L 180.105469 14.492188 C 179.933594 15.09375 179.625 15.5625 179.1875 15.898438 C 178.757812 16.226562 178.199219 16.386719 177.527344 16.386719 C 176.675781 16.386719 176.003906 16.128906 175.496094 15.605469 C 174.996094 15.074219 174.753906 14.332031 174.753906 13.378906 C 174.753906 12.402344 175.007812 11.648438 175.515625 11.113281 C 176.023438 10.566406 176.675781 10.292969 177.488281 10.292969 C 178.269531 10.292969 178.898438 10.5625 179.382812 11.09375 C 179.875 11.617188 180.125 12.359375 180.125 13.320312 C 180.125 13.390625 180.125 13.476562 180.125 13.59375 L 175.769531 13.59375 C 175.808594 14.234375 175.988281 14.722656 176.316406 15.058594 C 176.636719 15.398438 177.046875 15.566406 177.546875 15.566406 C 177.90625 15.566406 178.214844 15.476562 178.464844 15.292969 C 178.722656 15.097656 178.921875 14.796875 179.070312 14.375 Z M 175.828125 12.773438 L 179.089844 12.773438 C 179.050781 12.28125 178.921875 11.910156 178.71875 11.660156 C 178.40625 11.285156 177.996094 11.09375 177.488281 11.09375 C 177.027344 11.09375 176.648438 11.25 176.335938 11.5625 C 176.03125 11.867188 175.867188 12.265625 175.828125 12.773438 Z M 175.828125 12.773438 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 181.320312 16.25 L 181.320312 8.203125 L 182.296875 8.203125 L 182.296875 16.25 Z M 181.320312 16.25 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 39.792969 16.25 L 39.792969 8.203125 L 40.847656 8.203125 L 40.847656 16.25 Z M 39.792969 16.25 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 42.605469 16.25 L 42.605469 10.410156 L 43.503906 10.410156 L 43.503906 11.25 C 43.921875 10.617188 44.539062 10.292969 45.359375 10.292969 C 45.710938 10.292969 46.03125 10.359375 46.316406 10.488281 C 46.617188 10.605469 46.835938 10.773438 46.980469 10.976562 C 47.125 11.171875 47.226562 11.414062 47.292969 11.699219 C 47.332031 11.882812 47.351562 12.203125 47.351562 12.65625 L 47.351562 16.25 L 46.355469 16.25 L 46.355469 12.695312 C 46.355469 12.296875 46.316406 11.992188 46.238281 11.796875 C 46.160156 11.601562 46.023438 11.445312 45.828125 11.328125 C 45.632812 11.203125 45.40625 11.132812 45.144531 11.132812 C 44.726562 11.132812 44.363281 11.269531 44.050781 11.542969 C 43.75 11.804688 43.601562 12.3125 43.601562 13.066406 L 43.601562 16.25 Z M 42.605469 16.25 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 48.859375 18.476562 L 48.859375 10.410156 L 49.757812 10.410156 L 49.757812 11.171875 C 49.964844 10.875 50.199219 10.65625 50.460938 10.507812 C 50.734375 10.367188 51.066406 10.292969 51.457031 10.292969 C 51.949219 10.292969 52.386719 10.425781 52.765625 10.683594 C 53.140625 10.933594 53.429688 11.289062 53.625 11.757812 C 53.820312 12.226562 53.917969 12.734375 53.917969 13.28125 C 53.917969 13.882812 53.804688 14.421875 53.585938 14.902344 C 53.375 15.386719 53.070312 15.757812 52.667969 16.015625 C 52.261719 16.257812 51.832031 16.386719 51.378906 16.386719 C 51.050781 16.386719 50.757812 16.3125 50.5 16.171875 C 50.238281 16.03125 50.023438 15.855469 49.855469 15.644531 L 49.855469 18.476562 Z M 49.757812 13.359375 C 49.757812 14.117188 49.90625 14.679688 50.207031 15.039062 C 50.519531 15.390625 50.890625 15.566406 51.320312 15.566406 C 51.75 15.566406 52.121094 15.386719 52.433594 15.019531 C 52.757812 14.644531 52.921875 14.0625 52.921875 13.28125 C 52.921875 12.539062 52.765625 11.988281 52.453125 11.621094 C 52.152344 11.246094 51.789062 11.054688 51.359375 11.054688 C 50.941406 11.054688 50.570312 11.257812 50.246094 11.660156 C 49.917969 12.050781 49.757812 12.617188 49.757812 13.359375 Z M 49.757812 13.359375 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 58.941406 16.25 L 58.941406 15.390625 C 58.480469 16.054688 57.867188 16.386719 57.085938 16.386719 C 56.742188 16.386719 56.425781 16.316406 56.128906 16.191406 C 55.824219 16.050781 55.601562 15.878906 55.445312 15.683594 C 55.296875 15.488281 55.199219 15.25 55.152344 14.960938 C 55.113281 14.765625 55.09375 14.453125 55.09375 14.023438 L 55.09375 10.410156 L 56.070312 10.410156 L 56.070312 13.652344 C 56.070312 14.175781 56.09375 14.523438 56.148438 14.6875 C 56.195312 14.953125 56.324219 15.15625 56.519531 15.3125 C 56.722656 15.460938 56.976562 15.527344 57.28125 15.527344 C 57.578125 15.527344 57.855469 15.460938 58.121094 15.3125 C 58.378906 15.15625 58.558594 14.953125 58.667969 14.6875 C 58.769531 14.429688 58.824219 14.042969 58.824219 13.535156 L 58.824219 10.410156 L 59.820312 10.410156 L 59.820312 16.25 Z M 58.941406 16.25 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 63.53125 15.371094 L 63.667969 16.230469 C 63.382812 16.292969 63.132812 16.328125 62.925781 16.328125 C 62.5625 16.328125 62.28125 16.269531 62.085938 16.152344 C 61.890625 16.035156 61.746094 15.890625 61.65625 15.703125 C 61.578125 15.523438 61.539062 15.136719 61.539062 14.550781 L 61.539062 11.191406 L 60.816406 11.191406 L 60.816406 10.410156 L 61.539062 10.410156 L 61.539062 8.964844 L 62.535156 8.378906 L 62.535156 10.410156 L 63.53125 10.410156 L 63.53125 11.191406 L 62.535156 11.191406 L 62.535156 14.589844 C 62.535156 14.878906 62.546875 15.058594 62.574219 15.136719 C 62.613281 15.214844 62.671875 15.28125 62.75 15.332031 C 62.828125 15.386719 62.9375 15.410156 63.082031 15.410156 C 63.199219 15.410156 63.347656 15.398438 63.53125 15.371094 Z M 63.53125 15.371094 "/> +<path style="fill-rule:nonzero;fill:rgb(40%,74.901961%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 45 126 L 63 126 L 63 144 L 45 144 Z M 45 126 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 45 90 L 63 90 L 63 108 L 45 108 Z M 45 90 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 45 108 L 63 108 L 63 126 L 45 126 Z M 45 108 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 9 126 L 27 126 L 27 144 L 9 144 Z M 9 126 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 27 126 L 45 126 L 45 144 L 27 144 Z M 27 126 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(69.803922%,85.098039%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 9 90 L 27 90 L 27 108 L 9 108 Z M 9 90 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 9 108 L 27 108 L 27 126 L 9 126 Z M 9 108 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 27 90 L 45 90 L 45 108 L 27 108 Z M 27 90 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 27 108 L 45 108 L 45 126 L 27 126 Z M 27 108 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 114.300781 78.214844 L 114.808594 76.652344 C 115.964844 77.058594 116.808594 77.410156 117.347656 77.707031 C 117.199219 76.394531 117.132812 75.480469 117.132812 74.972656 L 118.714844 74.972656 C 118.683594 75.703125 118.601562 76.613281 118.460938 77.707031 C 119.210938 77.332031 120.082031 76.980469 121.058594 76.652344 L 121.566406 78.214844 C 120.628906 78.515625 119.714844 78.71875 118.832031 78.820312 C 119.269531 79.210938 119.90625 79.902344 120.726562 80.890625 L 119.398438 81.828125 C 118.96875 81.257812 118.464844 80.46875 117.894531 79.464844 C 117.347656 80.507812 116.867188 81.296875 116.46875 81.828125 L 115.160156 80.890625 C 116.019531 79.851562 116.628906 79.160156 116.996094 78.820312 C 116.042969 78.640625 115.144531 78.4375 114.300781 78.214844 Z M 114.300781 78.214844 "/> +<path style=" stroke:none;fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 228.523438 77.660156 L 217.878906 77.660156 L 217.878906 75.824219 L 228.523438 75.824219 Z M 228.523438 82.542969 L 217.878906 82.542969 L 217.878906 80.707031 L 228.523438 80.707031 Z M 228.523438 82.542969 "/> +<path style="fill-rule:nonzero;fill:rgb(69.803922%,85.098039%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 140.96875 72 L 158.96875 72 L 158.96875 90 L 140.96875 90 Z M 140.96875 72 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(69.803922%,85.098039%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 131.96875 81 L 149.96875 81 L 149.96875 99 L 131.96875 99 Z M 131.96875 81 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(69.803922%,85.098039%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 122.96875 90 L 140.96875 90 L 140.96875 108 L 122.96875 108 Z M 122.96875 90 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(40%,74.901961%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 140.96875 108 L 158.96875 108 L 158.96875 126 L 140.96875 126 Z M 140.96875 108 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(40%,74.901961%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 131.96875 117 L 149.96875 117 L 149.96875 135 L 131.96875 135 Z M 131.96875 117 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(40%,74.901961%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 122.96875 126 L 140.96875 126 L 140.96875 144 L 122.96875 144 Z M 122.96875 126 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(40%,74.901961%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 258.575 113.3625 L 276.575 113.3625 L 276.575 131.3625 L 258.575 131.3625 Z M 258.575 113.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 258.575 77.3625 L 276.575 77.3625 L 276.575 95.3625 L 258.575 95.3625 Z M 258.575 77.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 258.575 95.3625 L 276.575 95.3625 L 276.575 113.3625 L 258.575 113.3625 Z M 258.575 95.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 222.575 113.3625 L 240.575 113.3625 L 240.575 131.3625 L 222.575 131.3625 Z M 222.575 113.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 240.575 113.3625 L 258.575 113.3625 L 258.575 131.3625 L 240.575 131.3625 Z M 240.575 113.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 222.575 77.3625 L 240.575 77.3625 L 240.575 95.3625 L 222.575 95.3625 Z M 222.575 77.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 222.575 95.3625 L 240.575 95.3625 L 240.575 113.3625 L 222.575 113.3625 Z M 222.575 95.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 240.575 77.3625 L 258.575 77.3625 L 258.575 95.3625 L 240.575 95.3625 Z M 240.575 77.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 240.575 95.3625 L 258.575 95.3625 L 258.575 113.3625 L 240.575 113.3625 Z M 240.575 95.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 249.575 122.3625 L 267.575 122.3625 L 267.575 140.3625 L 249.575 140.3625 Z M 249.575 122.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 249.575 86.3625 L 267.575 86.3625 L 267.575 104.3625 L 249.575 104.3625 Z M 249.575 86.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 249.575 104.3625 L 267.575 104.3625 L 267.575 122.3625 L 249.575 122.3625 Z M 249.575 104.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 213.575 122.3625 L 231.575 122.3625 L 231.575 140.3625 L 213.575 140.3625 Z M 213.575 122.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 231.575 122.3625 L 249.575 122.3625 L 249.575 140.3625 L 231.575 140.3625 Z M 231.575 122.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(69.803922%,85.098039%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 213.575 86.3625 L 231.575 86.3625 L 231.575 104.3625 L 213.575 104.3625 Z M 213.575 86.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 213.575 104.3625 L 231.575 104.3625 L 231.575 122.3625 L 213.575 122.3625 Z M 213.575 104.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 231.575 86.3625 L 249.575 86.3625 L 249.575 104.3625 L 231.575 104.3625 Z M 231.575 86.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +<path style="fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 231.575 104.3625 L 249.575 104.3625 L 249.575 122.3625 L 231.575 122.3625 Z M 231.575 104.3625 " transform="matrix(1.25,0,0,1.25,-10,-55)"/> +</g> +</svg> diff --git a/_resources/d00c7365a714367186d44f165573aa33.png b/_resources/d00c7365a714367186d44f165573aa33.png new file mode 100644 index 0000000..0fbee44 Binary files /dev/null and b/_resources/d00c7365a714367186d44f165573aa33.png differ diff --git a/_resources/d2d5237d7c278a8fe8ffbe3175b6a7bc.png b/_resources/d2d5237d7c278a8fe8ffbe3175b6a7bc.png new file mode 100644 index 0000000..346b16f Binary files /dev/null and b/_resources/d2d5237d7c278a8fe8ffbe3175b6a7bc.png differ diff --git a/_resources/d575f021de579d10e3855c763198e7bc.png b/_resources/d575f021de579d10e3855c763198e7bc.png new file mode 100644 index 0000000..c421aac Binary files /dev/null and b/_resources/d575f021de579d10e3855c763198e7bc.png differ diff --git a/_resources/d74445c2bc5cf347c94f572c4787c354.png b/_resources/d74445c2bc5cf347c94f572c4787c354.png new file mode 100644 index 0000000..d0c19e6 Binary files /dev/null and b/_resources/d74445c2bc5cf347c94f572c4787c354.png differ diff --git a/_resources/d80f9b4acc1e1833c3fcb177f3a50dd8.png b/_resources/d80f9b4acc1e1833c3fcb177f3a50dd8.png new file mode 100644 index 0000000..938a59d Binary files /dev/null and b/_resources/d80f9b4acc1e1833c3fcb177f3a50dd8.png differ diff --git a/_resources/d9fd10fbec213e8ae877a457a76bfe82.png b/_resources/d9fd10fbec213e8ae877a457a76bfe82.png new file mode 100644 index 0000000..9f9e978 Binary files /dev/null and b/_resources/d9fd10fbec213e8ae877a457a76bfe82.png differ diff --git a/_resources/da8ec8c3f3b5290b4f2ec2c5fb3a7122.png b/_resources/da8ec8c3f3b5290b4f2ec2c5fb3a7122.png new file mode 100644 index 0000000..0669a7f Binary files /dev/null and b/_resources/da8ec8c3f3b5290b4f2ec2c5fb3a7122.png differ diff --git a/_resources/da9fb26de9169d4567f32995e6380343.png b/_resources/da9fb26de9169d4567f32995e6380343.png new file mode 100644 index 0000000..322e0b9 Binary files /dev/null and b/_resources/da9fb26de9169d4567f32995e6380343.png differ diff --git a/_resources/data-augmentation.png b/_resources/data-augmentation.png new file mode 100644 index 0000000..f387003 Binary files /dev/null and b/_resources/data-augmentation.png differ diff --git a/_resources/data-dist.png b/_resources/data-dist.png new file mode 100644 index 0000000..0ad01d3 Binary files /dev/null and b/_resources/data-dist.png differ diff --git a/_resources/data-hand-engineering.png b/_resources/data-hand-engineering.png new file mode 100644 index 0000000..46ce502 Binary files /dev/null and b/_resources/data-hand-engineering.png differ diff --git a/_resources/db70a1d3769f8483876fb44e83fbc35c.png b/_resources/db70a1d3769f8483876fb44e83fbc35c.png new file mode 100644 index 0000000..ca877ed Binary files /dev/null and b/_resources/db70a1d3769f8483876fb44e83fbc35c.png differ diff --git a/_resources/de4bf8596cb1518879ba2589540b3c7d.png b/_resources/de4bf8596cb1518879ba2589540b3c7d.png new file mode 100644 index 0000000..7f63f9b Binary files /dev/null and b/_resources/de4bf8596cb1518879ba2589540b3c7d.png differ diff --git a/_resources/dropout.jpeg b/_resources/dropout.jpeg new file mode 100644 index 0000000..1d16328 Binary files /dev/null and b/_resources/dropout.jpeg differ diff --git a/_resources/e3bb1a88acbda798341b1985f38d888c.png b/_resources/e3bb1a88acbda798341b1985f38d888c.png new file mode 100644 index 0000000..1e3ea41 Binary files /dev/null and b/_resources/e3bb1a88acbda798341b1985f38d888c.png differ diff --git a/_resources/e46733bcebbc39b3852081970b1ad9cd.png b/_resources/e46733bcebbc39b3852081970b1ad9cd.png new file mode 100644 index 0000000..0680513 Binary files /dev/null and b/_resources/e46733bcebbc39b3852081970b1ad9cd.png differ diff --git a/_resources/e9bdd75ea38db93bd06929d6d2371de2.png b/_resources/e9bdd75ea38db93bd06929d6d2371de2.png new file mode 100644 index 0000000..305d438 Binary files /dev/null and b/_resources/e9bdd75ea38db93bd06929d6d2371de2.png differ diff --git a/_resources/e9effef3a9891b908b2197d351856eff.png b/_resources/e9effef3a9891b908b2197d351856eff.png new file mode 100644 index 0000000..55c948a Binary files /dev/null and b/_resources/e9effef3a9891b908b2197d351856eff.png differ diff --git a/_resources/eb4e1a22065b0f7b00096573ca4f7cff.png b/_resources/eb4e1a22065b0f7b00096573ca4f7cff.png new file mode 100644 index 0000000..a421a07 Binary files /dev/null and b/_resources/eb4e1a22065b0f7b00096573ca4f7cff.png differ diff --git a/_resources/ec27c52f379456fd1ccdcd586b69c7de.png b/_resources/ec27c52f379456fd1ccdcd586b69c7de.png new file mode 100644 index 0000000..dbe1355 Binary files /dev/null and b/_resources/ec27c52f379456fd1ccdcd586b69c7de.png differ diff --git a/_resources/edge-detect-v.png b/_resources/edge-detect-v.png new file mode 100644 index 0000000..c1db259 Binary files /dev/null and b/_resources/edge-detect-v.png differ diff --git a/_resources/embedding_debiased.svg b/_resources/embedding_debiased.svg new file mode 100644 index 0000000..32725e1 --- /dev/null +++ b/_resources/embedding_debiased.svg @@ -0,0 +1,82 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='155.028542pt' height='44.211292pt' viewBox='-.239051 -.228145 155.028542 44.211292'> +<defs> +<path id='g3-50' d='M2.247572-1.625903C2.375093-1.745455 2.709838-2.008468 2.83736-2.12005C3.331507-2.574346 3.801743-3.012702 3.801743-3.737983C3.801743-4.686426 3.004732-5.300125 2.008468-5.300125C1.052055-5.300125 .422416-4.574844 .422416-3.865504C.422416-3.474969 .73325-3.419178 .844832-3.419178C1.012204-3.419178 1.259278-3.53873 1.259278-3.841594C1.259278-4.25604 .860772-4.25604 .765131-4.25604C.996264-4.837858 1.530262-5.037111 1.920797-5.037111C2.662017-5.037111 3.044583-4.407472 3.044583-3.737983C3.044583-2.909091 2.462765-2.303362 1.522291-1.338979L.518057-.302864C.422416-.215193 .422416-.199253 .422416 0H3.57061L3.801743-1.42665H3.55467C3.53076-1.267248 3.466999-.868742 3.371357-.71731C3.323537-.653549 2.717808-.653549 2.590286-.653549H1.171606L2.247572-1.625903Z'/> +<path id='g0-0' d='M7.878456-2.749689C8.081694-2.749689 8.296887-2.749689 8.296887-2.988792S8.081694-3.227895 7.878456-3.227895H1.41071C1.207472-3.227895 .992279-3.227895 .992279-2.988792S1.207472-2.749689 1.41071-2.749689H7.878456Z'/> +<path id='g0-1' d='M2.295392-2.988792C2.295392-3.335492 2.008468-3.622416 1.661768-3.622416S1.028144-3.335492 1.028144-2.988792S1.315068-2.355168 1.661768-2.355168S2.295392-2.642092 2.295392-2.988792Z'/> +<path id='g0-3' d='M3.287671-5.104857C3.299626-5.272229 3.299626-5.559153 2.988792-5.559153C2.797509-5.559153 2.642092-5.403736 2.677958-5.248319V-5.092902L2.84533-3.239851L1.315068-4.351681C1.207472-4.411457 1.183562-4.435367 1.099875-4.435367C.932503-4.435367 .777086-4.267995 .777086-4.100623C.777086-3.90934 .896638-3.861519 1.016189-3.801743L2.713823-2.988792L1.06401-2.187796C.872727-2.092154 .777086-2.044334 .777086-1.865006S.932503-1.530262 1.099875-1.530262C1.183562-1.530262 1.207472-1.530262 1.506351-1.75741L2.84533-2.725778L2.666002-.71731C2.666002-.466252 2.881196-.406476 2.976837-.406476C3.120299-.406476 3.299626-.490162 3.299626-.71731L3.120299-2.725778L4.65056-1.613948C4.758157-1.554172 4.782067-1.530262 4.865753-1.530262C5.033126-1.530262 5.188543-1.697634 5.188543-1.865006C5.188543-2.044334 5.080946-2.10411 4.937484-2.175841C4.220174-2.534496 4.196264-2.534496 3.251806-2.976837L4.901619-3.777833C5.092902-3.873474 5.188543-3.921295 5.188543-4.100623S5.033126-4.435367 4.865753-4.435367C4.782067-4.435367 4.758157-4.435367 4.459278-4.208219L3.120299-3.239851L3.287671-5.104857Z'/> +<path id='g0-107' d='M2.056289-8.5599C2.056289-8.751183 2.056289-8.966376 1.817186-8.966376S1.578082-8.703362 1.578082-8.524035V2.546451C1.578082 2.737733 1.578082 2.988792 1.817186 2.988792S2.056289 2.773599 2.056289 2.582316V-8.5599ZM4.387547-8.524035C4.387547-8.715318 4.387547-8.966376 4.148443-8.966376S3.90934-8.751183 3.90934-8.5599V2.582316C3.90934 2.773599 3.90934 2.988792 4.148443 2.988792S4.387547 2.725778 4.387547 2.546451V-8.524035Z'/> +<path id='g4-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g1-97' d='M3.124284-3.036613C3.052553-3.172105 2.82142-3.514819 2.335243-3.514819C1.3868-3.514819 .342715-2.406974 .342715-1.227397C.342715-.398506 .876712 .079701 1.490411 .079701C2.000498 .079701 2.438854-.326775 2.582316-.486177C2.725778 .063761 3.267746 .079701 3.363387 .079701C3.730012 .079701 3.913325-.223163 3.977086-.358655C4.136488-.645579 4.24807-1.107846 4.24807-1.139726C4.24807-1.187547 4.216189-1.243337 4.120548-1.243337S4.008966-1.195517 3.961146-.996264C3.849564-.557908 3.698132-.143462 3.387298-.143462C3.203985-.143462 3.132254-.294894 3.132254-.518057C3.132254-.653549 3.203985-.924533 3.251806-1.123786S3.419178-1.801245 3.451059-1.944707L3.610461-2.550436C3.650311-2.741719 3.737983-3.076463 3.737983-3.116314C3.737983-3.299626 3.58655-3.363387 3.482939-3.363387C3.363387-3.363387 3.164134-3.283686 3.124284-3.036613ZM2.582316-.860772C2.183811-.310834 1.769365-.143462 1.514321-.143462C1.147696-.143462 .964384-.478207 .964384-.892653C.964384-1.267248 1.179577-2.12005 1.354919-2.470735C1.586052-2.956912 1.976588-3.291656 2.343213-3.291656C2.86127-3.291656 3.012702-2.709838 3.012702-2.614197C3.012702-2.582316 2.81345-1.801245 2.765629-1.594022C2.662017-1.219427 2.662017-1.203487 2.582316-.860772Z'/> +<path id='g1-98' d='M1.944707-5.292154C1.952677-5.308095 1.976588-5.411706 1.976588-5.419676C1.976588-5.459527 1.944707-5.531258 1.849066-5.531258C1.817186-5.531258 1.570112-5.507347 1.3868-5.491407L.940473-5.459527C.765131-5.443587 .68543-5.435616 .68543-5.292154C.68543-5.180573 .797011-5.180573 .892653-5.180573C1.275218-5.180573 1.275218-5.132752 1.275218-5.061021C1.275218-5.0132 1.195517-4.694396 1.147696-4.511083L.454296-1.737484C.390535-1.466501 .390535-1.346949 .390535-1.211457C.390535-.390535 .892653 .079701 1.506351 .079701C2.486675 .079701 3.506849-1.052055 3.506849-2.207721C3.506849-2.996762 2.996762-3.514819 2.359153-3.514819C1.912827-3.514819 1.570112-3.227895 1.39477-3.076463L1.944707-5.292154ZM1.506351-.143462C1.219427-.143462 .932503-.366625 .932503-.948443C.932503-1.163636 .964384-1.362889 1.060025-1.745455C1.115816-1.976588 1.171606-2.199751 1.235367-2.430884C1.275218-2.574346 1.275218-2.590286 1.370859-2.709838C1.641843-3.044583 2.000498-3.291656 2.335243-3.291656C2.733748-3.291656 2.885181-2.901121 2.885181-2.542466C2.885181-2.247572 2.709838-1.39477 2.470735-.924533C2.263512-.494147 1.880946-.143462 1.506351-.143462Z'/> +<path id='g1-99' d='M3.259776-3.052553C2.933001-3.012702 2.82939-2.765629 2.82939-2.606227C2.82939-2.375093 3.036613-2.311333 3.140224-2.311333C3.180075-2.311333 3.58655-2.343213 3.58655-2.82939S3.060523-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.541968 .868742 .079701 1.753425 .079701C3.012702 .079701 3.674222-.72528 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.769365-.143462C1.338979-.143462 .996264-.406476 .996264-1.012204C.996264-1.362889 1.155666-2.207721 1.530262-2.701868C1.880946-3.148194 2.279452-3.291656 2.590286-3.291656C2.685928-3.291656 3.052553-3.283686 3.259776-3.052553Z'/> +<path id='g1-100' d='M4.28792-5.292154C4.29589-5.308095 4.319801-5.411706 4.319801-5.419676C4.319801-5.459527 4.28792-5.531258 4.192279-5.531258C4.160399-5.531258 3.913325-5.507347 3.730012-5.491407L3.283686-5.459527C3.108344-5.443587 3.028643-5.435616 3.028643-5.292154C3.028643-5.180573 3.140224-5.180573 3.235866-5.180573C3.618431-5.180573 3.618431-5.132752 3.618431-5.061021C3.618431-5.0132 3.55467-4.750187 3.514819-4.590785L3.124284-3.036613C3.052553-3.172105 2.82142-3.514819 2.335243-3.514819C1.3868-3.514819 .342715-2.406974 .342715-1.227397C.342715-.398506 .876712 .079701 1.490411 .079701C2.000498 .079701 2.438854-.326775 2.582316-.486177C2.725778 .063761 3.267746 .079701 3.363387 .079701C3.730012 .079701 3.913325-.223163 3.977086-.358655C4.136488-.645579 4.24807-1.107846 4.24807-1.139726C4.24807-1.187547 4.216189-1.243337 4.120548-1.243337S4.008966-1.195517 3.961146-.996264C3.849564-.557908 3.698132-.143462 3.387298-.143462C3.203985-.143462 3.132254-.294894 3.132254-.518057C3.132254-.669489 3.156164-.757161 3.180075-.860772L4.28792-5.292154ZM2.582316-.860772C2.183811-.310834 1.769365-.143462 1.514321-.143462C1.147696-.143462 .964384-.478207 .964384-.892653C.964384-1.267248 1.179577-2.12005 1.354919-2.470735C1.586052-2.956912 1.976588-3.291656 2.343213-3.291656C2.86127-3.291656 3.012702-2.709838 3.012702-2.614197C3.012702-2.582316 2.81345-1.801245 2.765629-1.594022C2.662017-1.219427 2.662017-1.203487 2.582316-.860772Z'/> +<path id='g1-101' d='M1.601993-1.817186C1.777335-1.817186 2.375093-1.825156 2.797509-1.976588C3.490909-2.215691 3.514819-2.693898 3.514819-2.81345C3.514819-3.259776 3.092403-3.514819 2.574346-3.514819C1.673724-3.514819 .390535-2.805479 .390535-1.39477C.390535-.581818 .884682 .079701 1.761395 .079701C3.004732 .079701 3.674222-.71731 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.777335-.143462C1.195517-.143462 1.020174-.637609 1.020174-1.083935C1.020174-1.323039 1.091905-1.681694 1.123786-1.817186H1.601993ZM1.187547-2.040349C1.44259-3.012702 2.16787-3.291656 2.574346-3.291656C2.893151-3.291656 3.196015-3.132254 3.196015-2.81345C3.196015-2.040349 1.888917-2.040349 1.554172-2.040349H1.187547Z'/> +<path id='g1-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g1-109' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.745455-1.928767 1.793275-2.12005 1.809215-2.199751C1.825156-2.239601 2.088169-2.757659 2.438854-3.020672C2.709838-3.227895 2.972852-3.291656 3.196015-3.291656C3.490909-3.291656 3.650311-3.116314 3.650311-2.749689C3.650311-2.558406 3.602491-2.375093 3.514819-2.016438C3.459029-1.809215 3.323537-1.275218 3.275716-1.060025L3.156164-.581818C3.116314-.446326 3.060523-.207223 3.060523-.167372C3.060523 .01594 3.211955 .079701 3.315567 .079701C3.459029 .079701 3.57858-.01594 3.634371-.111582C3.658281-.159402 3.722042-.430386 3.761893-.597758L3.945205-1.307098C3.969116-1.42665 4.048817-1.729514 4.072727-1.849066C4.184309-2.279452 4.184309-2.287422 4.367621-2.550436C4.630635-2.940971 5.00523-3.291656 5.539228-3.291656C5.826152-3.291656 5.993524-3.124284 5.993524-2.749689C5.993524-2.311333 5.65878-1.39477 5.507347-1.012204C5.427646-.804981 5.403736-.749191 5.403736-.597758C5.403736-.143462 5.778331 .079701 6.121046 .079701C6.902117 .079701 7.228892-1.036115 7.228892-1.139726C7.228892-1.219427 7.165131-1.243337 7.10934-1.243337C7.013699-1.243337 6.997758-1.187547 6.973848-1.107846C6.782565-.446326 6.447821-.143462 6.144956-.143462C6.017435-.143462 5.953674-.223163 5.953674-.406476S6.017435-.765131 6.097136-.964384C6.216687-1.267248 6.567372-2.183811 6.567372-2.630137C6.567372-3.227895 6.152927-3.514819 5.579078-3.514819C5.029141-3.514819 4.574844-3.227895 4.216189-2.733748C4.152428-3.371357 3.642341-3.514819 3.227895-3.514819C2.86127-3.514819 2.375093-3.387298 1.936737-2.81345C1.880946-3.291656 1.498381-3.514819 1.123786-3.514819C.844832-3.514819 .645579-3.347447 .510087-3.076463C.318804-2.701868 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g1-110' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.833126-2.279452 1.833126-2.287422 2.016438-2.550436C2.279452-2.940971 2.654047-3.291656 3.188045-3.291656C3.474969-3.291656 3.642341-3.124284 3.642341-2.749689C3.642341-2.311333 3.307597-1.40274 3.156164-1.012204C3.052553-.749191 3.052553-.70137 3.052553-.597758C3.052553-.143462 3.427148 .079701 3.769863 .079701C4.550934 .079701 4.877709-1.036115 4.877709-1.139726C4.877709-1.219427 4.813948-1.243337 4.758157-1.243337C4.662516-1.243337 4.646575-1.187547 4.622665-1.107846C4.431382-.454296 4.096638-.143462 3.793773-.143462C3.666252-.143462 3.602491-.223163 3.602491-.406476S3.666252-.765131 3.745953-.964384C3.865504-1.267248 4.216189-2.183811 4.216189-2.630137C4.216189-3.227895 3.801743-3.514819 3.227895-3.514819C2.582316-3.514819 2.16787-3.124284 1.936737-2.82142C1.880946-3.259776 1.530262-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.318804-2.709838 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g1-111' d='M3.969116-2.13599C3.969116-2.917061 3.411208-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.486177 .924533 .079701 1.737484 .079701C2.87721 .079701 3.969116-1.036115 3.969116-2.13599ZM1.745455-.143462C1.466501-.143462 .996264-.286924 .996264-1.020174C.996264-1.346949 1.147696-2.207721 1.530262-2.701868C1.920797-3.203985 2.359153-3.291656 2.574346-3.291656C2.901121-3.291656 3.323537-3.092403 3.323537-2.422914C3.323537-2.10411 3.180075-1.346949 2.87721-.868742C2.582316-.414446 2.14396-.143462 1.745455-.143462Z'/> +<path id='g1-112' d='M.414446 .964384C.350685 1.219427 .334745 1.283188 .01594 1.283188C-.095641 1.283188-.191283 1.283188-.191283 1.43462C-.191283 1.506351-.119552 1.546202-.079701 1.546202C0 1.546202 .03188 1.522291 .621669 1.522291C1.195517 1.522291 1.362889 1.546202 1.41868 1.546202C1.45056 1.546202 1.570112 1.546202 1.570112 1.39477C1.570112 1.283188 1.458531 1.283188 1.362889 1.283188C.980324 1.283188 .980324 1.235367 .980324 1.163636C.980324 1.107846 1.123786 .541968 1.362889-.390535C1.466501-.207223 1.713574 .079701 2.14396 .079701C3.124284 .079701 4.144458-1.052055 4.144458-2.207721C4.144458-2.996762 3.634371-3.514819 2.996762-3.514819C2.518555-3.514819 2.13599-3.188045 1.904857-2.948941C1.737484-3.514819 1.203487-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.326775-2.725778 .239103-2.319303 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.629639-2.83736 .773101-3.291656 1.099875-3.291656C1.299128-3.291656 1.354919-3.108344 1.354919-2.917061C1.354919-2.83736 1.323039-2.646077 1.307098-2.582316L.414446 .964384ZM1.880946-2.454795C1.920797-2.590286 1.920797-2.606227 2.040349-2.749689C2.343213-3.108344 2.685928-3.291656 2.972852-3.291656C3.371357-3.291656 3.52279-2.901121 3.52279-2.542466C3.52279-2.247572 3.347447-1.39477 3.108344-.924533C2.901121-.494147 2.518555-.143462 2.14396-.143462C1.601993-.143462 1.474471-.765131 1.474471-.820922C1.474471-.836862 1.490411-.924533 1.498381-.948443L1.880946-2.454795Z'/> +<path id='g1-115' d='M3.211955-2.996762C3.028643-2.964882 2.86127-2.82142 2.86127-2.622167C2.86127-2.478705 2.956912-2.375093 3.132254-2.375093C3.251806-2.375093 3.498879-2.462765 3.498879-2.82142C3.498879-3.315567 2.980822-3.514819 2.486675-3.514819C1.41868-3.514819 1.083935-2.757659 1.083935-2.351183C1.083935-2.271482 1.083935-1.984558 1.378829-1.761395C1.562142-1.617933 1.697634-1.594022 2.11208-1.514321C2.391034-1.458531 2.84533-1.378829 2.84533-.964384C2.84533-.757161 2.693898-.494147 2.470735-.342715C2.175841-.151432 1.785305-.143462 1.657783-.143462C1.466501-.143462 .924533-.175342 .72528-.494147C1.131756-.510087 1.187547-.836862 1.187547-.932503C1.187547-1.171606 .972354-1.227397 .876712-1.227397C.749191-1.227397 .422416-1.131756 .422416-.6934C.422416-.223163 .916563 .079701 1.657783 .079701C3.044583 .079701 3.339477-.900623 3.339477-1.235367C3.339477-1.952677 2.558406-2.10411 2.263512-2.1599C1.880946-2.231631 1.570112-2.287422 1.570112-2.622167C1.570112-2.765629 1.705604-3.291656 2.478705-3.291656C2.781569-3.291656 3.092403-3.203985 3.211955-2.996762Z'/> +<path id='g1-116' d='M1.761395-3.172105H2.542466C2.693898-3.172105 2.789539-3.172105 2.789539-3.323537C2.789539-3.435118 2.685928-3.435118 2.550436-3.435118H1.825156L2.11208-4.566874C2.14396-4.686426 2.14396-4.726276 2.14396-4.734247C2.14396-4.901619 2.016438-4.98132 1.880946-4.98132C1.609963-4.98132 1.554172-4.766127 1.466501-4.407472L1.219427-3.435118H.454296C.302864-3.435118 .199253-3.435118 .199253-3.283686C.199253-3.172105 .302864-3.172105 .438356-3.172105H1.155666L.67746-1.259278C.629639-1.060025 .557908-.781071 .557908-.669489C.557908-.191283 .948443 .079701 1.370859 .079701C2.223661 .079701 2.709838-1.044085 2.709838-1.139726C2.709838-1.227397 2.638107-1.243337 2.590286-1.243337C2.502615-1.243337 2.494645-1.211457 2.438854-1.091905C2.279452-.70934 1.880946-.143462 1.39477-.143462C1.227397-.143462 1.131756-.255044 1.131756-.518057C1.131756-.669489 1.155666-.757161 1.179577-.860772L1.761395-3.172105Z'/> +<path id='g2-101' d='M2.139975-2.773599C2.462765-2.773599 3.275716-2.797509 3.849564-3.012702C4.758157-3.359402 4.841843-4.052802 4.841843-4.267995C4.841843-4.794022 4.387547-5.272229 3.598506-5.272229C2.343213-5.272229 .537983-4.136488 .537983-2.008468C.537983-.753176 1.255293 .119552 2.343213 .119552C3.969116 .119552 4.99726-1.147696 4.99726-1.303113C4.99726-1.374844 4.925529-1.43462 4.877709-1.43462C4.841843-1.43462 4.829888-1.422665 4.722291-1.315068C3.957161-.298879 2.82142-.119552 2.367123-.119552C1.685679-.119552 1.327024-.657534 1.327024-1.542217C1.327024-1.709589 1.327024-2.008468 1.506351-2.773599H2.139975ZM1.566127-3.012702C2.080199-4.853798 3.21594-5.033126 3.598506-5.033126C4.124533-5.033126 4.483188-4.722291 4.483188-4.267995C4.483188-3.012702 2.570361-3.012702 2.068244-3.012702H1.566127Z'/> +<path id='g2-103' d='M4.040847-1.518306C3.993026-1.327024 3.969116-1.279203 3.813699-1.099875C3.323537-.466252 2.82142-.239103 2.450809-.239103C2.056289-.239103 1.685679-.549938 1.685679-1.374844C1.685679-2.008468 2.044334-3.347447 2.307347-3.88543C2.654047-4.554919 3.19203-5.033126 3.694147-5.033126C4.483188-5.033126 4.638605-4.052802 4.638605-3.981071L4.60274-3.813699L4.040847-1.518306ZM4.782067-4.483188C4.62665-4.829888 4.291905-5.272229 3.694147-5.272229C2.391034-5.272229 .908593-3.634371 .908593-1.853051C.908593-.609714 1.661768 0 2.426899 0C3.060523 0 3.622416-.502117 3.837609-.74122L3.574595 .334745C3.407223 .992279 3.335492 1.291158 2.905106 1.709589C2.414944 2.199751 1.960648 2.199751 1.697634 2.199751C1.338979 2.199751 1.0401 2.175841 .74122 2.080199C1.123786 1.972603 1.219427 1.637858 1.219427 1.506351C1.219427 1.315068 1.075965 1.123786 .812951 1.123786C.526027 1.123786 .215193 1.362889 .215193 1.75741C.215193 2.247572 .705355 2.438854 1.721544 2.438854C3.263761 2.438854 4.064757 1.446575 4.220174 .800996L5.547198-4.554919C5.583064-4.698381 5.583064-4.722291 5.583064-4.746202C5.583064-4.913574 5.451557-5.045081 5.272229-5.045081C4.985305-5.045081 4.817933-4.805978 4.782067-4.483188Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -61.066718)'> +<use x='56.413267' y='67.199433' xlink:href='#g2-101'/> +<use x='61.838707' y='62.263247' xlink:href='#g1-98'/> +<use x='65.461292' y='62.263247' xlink:href='#g1-105'/> +<use x='68.344432' y='62.263247' xlink:href='#g1-97'/> +<use x='72.842441' y='62.263247' xlink:href='#g1-115'/> +<rect x='77.266434' y='61.864748' height='.3985' width='2.540522'/> +<use x='79.806956' y='62.263247' xlink:href='#g1-99'/> +<use x='83.474735' y='62.263247' xlink:href='#g1-111'/> +<use x='87.567785' y='62.263247' xlink:href='#g1-109'/> +<use x='95.058311' y='62.263247' xlink:href='#g1-112'/> +<use x='99.321088' y='62.263247' xlink:href='#g1-111'/> +<use x='103.414138' y='62.263247' xlink:href='#g1-110'/> +<use x='108.55234' y='62.263247' xlink:href='#g1-101'/> +<use x='112.480724' y='62.263247' xlink:href='#g1-110'/> +<use x='117.618927' y='62.263247' xlink:href='#g1-116'/> +<use x='124.495909' y='67.199433' xlink:href='#g4-61'/> +<use x='139.430839' y='59.111675' xlink:href='#g2-101'/> +<use x='147.512943' y='59.111675' xlink:href='#g0-1'/> +<use x='153.490496' y='59.111675' xlink:href='#g2-103'/> +<rect x='138.116904' y='63.971548' height='.478187' width='22.72177'/> +<use x='138.116904' y='75.400096' xlink:href='#g0-107'/> +<use x='144.09451' y='75.400096' xlink:href='#g2-103'/> +<use x='150.128767' y='75.400096' xlink:href='#g0-107'/> +<use x='156.106374' y='71.282222' xlink:href='#g3-50'/> +<use x='156.106374' y='78.331273' xlink:href='#g3-50'/> +<use x='164.69085' y='67.199433' xlink:href='#g0-3'/> +<use x='173.325121' y='67.199433' xlink:href='#g2-103'/> +<use x='56.413267' y='92.844931' xlink:href='#g2-101'/> +<use x='61.838707' y='87.908746' xlink:href='#g1-100'/> +<use x='66.196025' y='87.908746' xlink:href='#g1-101'/> +<use x='70.124408' y='87.908746' xlink:href='#g1-98'/> +<use x='73.746993' y='87.908746' xlink:href='#g1-105'/> +<use x='76.630133' y='87.908746' xlink:href='#g1-97'/> +<use x='81.128143' y='87.908746' xlink:href='#g1-115'/> +<use x='85.044061' y='87.908746' xlink:href='#g1-101'/> +<use x='88.972445' y='87.908746' xlink:href='#g1-100'/> +<use x='97.148723' y='92.844931' xlink:href='#g4-61'/> +<use x='109.574204' y='92.844931' xlink:href='#g2-101'/> +<use x='117.656308' y='92.844931' xlink:href='#g0-0'/> +<use x='129.611468' y='92.844931' xlink:href='#g2-101'/> +<use x='135.036908' y='87.908746' xlink:href='#g1-98'/> +<use x='138.659493' y='87.908746' xlink:href='#g1-105'/> +<use x='141.542633' y='87.908746' xlink:href='#g1-97'/> +<use x='146.040643' y='87.908746' xlink:href='#g1-115'/> +<rect x='150.464635' y='87.510246' height='.3985' width='2.540522'/> +<use x='153.005157' y='87.908746' xlink:href='#g1-99'/> +<use x='156.672936' y='87.908746' xlink:href='#g1-111'/> +<use x='160.765986' y='87.908746' xlink:href='#g1-109'/> +<use x='168.256512' y='87.908746' xlink:href='#g1-112'/> +<use x='172.51929' y='87.908746' xlink:href='#g1-111'/> +<use x='176.612339' y='87.908746' xlink:href='#g1-110'/> +<use x='181.750541' y='87.908746' xlink:href='#g1-101'/> +<use x='185.678925' y='87.908746' xlink:href='#g1-110'/> +<use x='190.817128' y='87.908746' xlink:href='#g1-116'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/encoder-decoder.png b/_resources/encoder-decoder.png new file mode 100644 index 0000000..e805328 Binary files /dev/null and b/_resources/encoder-decoder.png differ diff --git a/_resources/end-to-end.png b/_resources/end-to-end.png new file mode 100644 index 0000000..3ba419f Binary files /dev/null and b/_resources/end-to-end.png differ diff --git a/_resources/equalize.png b/_resources/equalize.png new file mode 100644 index 0000000..cf10aa9 Binary files /dev/null and b/_resources/equalize.png differ diff --git a/_resources/ewa-temp-plot1.png b/_resources/ewa-temp-plot1.png new file mode 100644 index 0000000..3d85aba Binary files /dev/null and b/_resources/ewa-temp-plot1.png differ diff --git a/_resources/ewa-temp-plot2.png b/_resources/ewa-temp-plot2.png new file mode 100644 index 0000000..5b6ecec Binary files /dev/null and b/_resources/ewa-temp-plot2.png differ diff --git a/_resources/ewa-temp-plot3.png b/_resources/ewa-temp-plot3.png new file mode 100644 index 0000000..6f83247 Binary files /dev/null and b/_resources/ewa-temp-plot3.png differ diff --git a/_resources/ewa-temp1.svg b/_resources/ewa-temp1.svg new file mode 100644 index 0000000..6b431b7 --- /dev/null +++ b/_resources/ewa-temp1.svg @@ -0,0 +1,65 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='67.433123pt' height='110.414858pt' viewBox='-.239051 -.234468 67.433123 110.414858'> +<defs> +<path id='g0-14' d='M3.753923-1.992528C3.753923-2.909091 3.020672-3.642341 2.11208-3.642341S.470237-2.909091 .470237-1.992528S1.203487-.342715 2.11208-.342715S3.753923-1.075965 3.753923-1.992528ZM2.11208-.70934C1.40274-.70934 .836862-1.275218 .836862-1.992528S1.40274-3.275716 2.11208-3.275716S3.387298-2.709838 3.387298-1.992528S2.82142-.70934 2.11208-.70934Z'/> +<path id='g3-48' d='M5.355915-3.825654C5.355915-4.817933 5.296139-5.786301 4.865753-6.694894C4.375592-7.687173 3.514819-7.950187 2.929016-7.950187C2.235616-7.950187 1.3868-7.603487 .944458-6.611208C.609714-5.858032 .490162-5.116812 .490162-3.825654C.490162-2.666002 .573848-1.793275 1.004234-.944458C1.470486-.035866 2.295392 .251059 2.917061 .251059C3.957161 .251059 4.554919-.37061 4.901619-1.06401C5.332005-1.960648 5.355915-3.132254 5.355915-3.825654ZM2.917061 .011955C2.534496 .011955 1.75741-.203238 1.530262-1.506351C1.398755-2.223661 1.398755-3.132254 1.398755-3.969116C1.398755-4.94944 1.398755-5.834122 1.590037-6.539477C1.793275-7.340473 2.402989-7.711083 2.917061-7.711083C3.371357-7.711083 4.064757-7.436115 4.291905-6.40797C4.447323-5.726526 4.447323-4.782067 4.447323-3.969116C4.447323-3.16812 4.447323-2.259527 4.315816-1.530262C4.088667-.215193 3.335492 .011955 2.917061 .011955Z'/> +<path id='g3-52' d='M4.315816-7.782814C4.315816-8.009963 4.315816-8.069738 4.148443-8.069738C4.052802-8.069738 4.016936-8.069738 3.921295-7.926276L.32279-2.343213V-1.996513H3.466999V-.908593C3.466999-.466252 3.443088-.3467 2.570361-.3467H2.331258V0C2.606227-.02391 3.550685-.02391 3.88543-.02391S5.176588-.02391 5.451557 0V-.3467H5.212453C4.351681-.3467 4.315816-.466252 4.315816-.908593V-1.996513H5.523288V-2.343213H4.315816V-7.782814ZM3.526775-6.850311V-2.343213H.621669L3.526775-6.850311Z'/> +<path id='g3-53' d='M1.530262-6.850311C2.044334-6.682939 2.462765-6.670984 2.594271-6.670984C3.945205-6.670984 4.805978-7.663263 4.805978-7.830635C4.805978-7.878456 4.782067-7.938232 4.710336-7.938232C4.686426-7.938232 4.662516-7.938232 4.554919-7.890411C3.88543-7.603487 3.311582-7.567621 3.000747-7.567621C2.211706-7.567621 1.649813-7.806725 1.422665-7.902366C1.338979-7.938232 1.315068-7.938232 1.303113-7.938232C1.207472-7.938232 1.207472-7.866501 1.207472-7.675218V-4.124533C1.207472-3.90934 1.207472-3.837609 1.350934-3.837609C1.41071-3.837609 1.422665-3.849564 1.542217-3.993026C1.876961-4.483188 2.438854-4.770112 3.036613-4.770112C3.670237-4.770112 3.981071-4.184309 4.076712-3.981071C4.27995-3.514819 4.291905-2.929016 4.291905-2.47472S4.291905-1.338979 3.957161-.800996C3.694147-.37061 3.227895-.071731 2.701868-.071731C1.912827-.071731 1.135741-.609714 .920548-1.482441C.980324-1.458531 1.052055-1.446575 1.111831-1.446575C1.315068-1.446575 1.637858-1.566127 1.637858-1.972603C1.637858-2.307347 1.41071-2.49863 1.111831-2.49863C.896638-2.49863 .585803-2.391034 .585803-1.924782C.585803-.908593 1.398755 .251059 2.725778 .251059C4.076712 .251059 5.260274-.884682 5.260274-2.402989C5.260274-3.825654 4.303861-5.009215 3.048568-5.009215C2.367123-5.009215 1.841096-4.710336 1.530262-4.375592V-6.850311Z'/> +<path id='g3-54' d='M1.470486-4.160399C1.470486-7.185056 2.940971-7.663263 3.58655-7.663263C4.016936-7.663263 4.447323-7.531756 4.674471-7.173101C4.531009-7.173101 4.076712-7.173101 4.076712-6.682939C4.076712-6.419925 4.25604-6.192777 4.566874-6.192777C4.865753-6.192777 5.068991-6.372105 5.068991-6.718804C5.068991-7.340473 4.614695-7.950187 3.574595-7.950187C2.068244-7.950187 .490162-6.40797 .490162-3.777833C.490162-.490162 1.924782 .251059 2.940971 .251059C4.244085 .251059 5.355915-.884682 5.355915-2.438854C5.355915-4.028892 4.244085-5.092902 3.048568-5.092902C1.984558-5.092902 1.590037-4.172354 1.470486-3.837609V-4.160399ZM2.940971-.071731C2.187796-.071731 1.829141-.74122 1.721544-.992279C1.613948-1.303113 1.494396-1.888917 1.494396-2.725778C1.494396-3.670237 1.924782-4.853798 3.000747-4.853798C3.658281-4.853798 4.004981-4.411457 4.184309-4.004981C4.375592-3.56264 4.375592-2.964882 4.375592-2.450809C4.375592-1.841096 4.375592-1.303113 4.148443-.848817C3.849564-.274969 3.419178-.071731 2.940971-.071731Z'/> +<path id='g3-57' d='M4.375592-3.478954C4.375592-.657534 3.120299-.071731 2.402989-.071731C2.116065-.071731 1.482441-.107597 1.183562-.526027H1.255293C1.338979-.502117 1.769365-.573848 1.769365-1.016189C1.769365-1.279203 1.590037-1.506351 1.279203-1.506351S.777086-1.303113 .777086-.992279C.777086-.251059 1.374844 .251059 2.414944 .251059C3.90934 .251059 5.355915-1.338979 5.355915-3.93325C5.355915-7.149191 4.016936-7.950187 2.964882-7.950187C1.649813-7.950187 .490162-6.850311 .490162-5.272229S1.601993-2.618182 2.797509-2.618182C3.682192-2.618182 4.136488-3.263761 4.375592-3.873474V-3.478954ZM2.84533-2.857285C2.092154-2.857285 1.769365-3.466999 1.661768-3.694147C1.470486-4.148443 1.470486-4.722291 1.470486-5.260274C1.470486-5.929763 1.470486-6.503611 1.78132-6.993773C1.996513-7.316563 2.319303-7.663263 2.964882-7.663263C3.646326-7.663263 3.993026-7.065504 4.112578-6.790535C4.351681-6.204732 4.351681-5.188543 4.351681-5.009215C4.351681-4.004981 3.897385-2.857285 2.84533-2.857285Z'/> +<path id='g3-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g2-48' d='M3.897385-2.542466C3.897385-3.395268 3.809714-3.913325 3.5467-4.423412C3.196015-5.124782 2.550436-5.300125 2.11208-5.300125C1.107846-5.300125 .74122-4.550934 .629639-4.327771C.342715-3.745953 .326775-2.956912 .326775-2.542466C.326775-2.016438 .350685-1.211457 .73325-.573848C1.099875 .01594 1.689664 .167372 2.11208 .167372C2.494645 .167372 3.180075 .047821 3.57858-.74122C3.873474-1.315068 3.897385-2.024408 3.897385-2.542466ZM2.11208-.055791C1.841096-.055791 1.291158-.183313 1.123786-1.020174C1.036115-1.474471 1.036115-2.223661 1.036115-2.638107C1.036115-3.188045 1.036115-3.745953 1.123786-4.184309C1.291158-4.99726 1.912827-5.076961 2.11208-5.076961C2.383064-5.076961 2.933001-4.941469 3.092403-4.216189C3.188045-3.777833 3.188045-3.180075 3.188045-2.638107C3.188045-2.16787 3.188045-1.45056 3.092403-1.004234C2.925031-.167372 2.375093-.055791 2.11208-.055791Z'/> +<path id='g2-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g2-50' d='M2.247572-1.625903C2.375093-1.745455 2.709838-2.008468 2.83736-2.12005C3.331507-2.574346 3.801743-3.012702 3.801743-3.737983C3.801743-4.686426 3.004732-5.300125 2.008468-5.300125C1.052055-5.300125 .422416-4.574844 .422416-3.865504C.422416-3.474969 .73325-3.419178 .844832-3.419178C1.012204-3.419178 1.259278-3.53873 1.259278-3.841594C1.259278-4.25604 .860772-4.25604 .765131-4.25604C.996264-4.837858 1.530262-5.037111 1.920797-5.037111C2.662017-5.037111 3.044583-4.407472 3.044583-3.737983C3.044583-2.909091 2.462765-2.303362 1.522291-1.338979L.518057-.302864C.422416-.215193 .422416-.199253 .422416 0H3.57061L3.801743-1.42665H3.55467C3.53076-1.267248 3.466999-.868742 3.371357-.71731C3.323537-.653549 2.717808-.653549 2.590286-.653549H1.171606L2.247572-1.625903Z'/> +<path id='g2-51' d='M2.016438-2.662017C2.646077-2.662017 3.044583-2.199751 3.044583-1.362889C3.044583-.366625 2.478705-.071731 2.056289-.071731C1.617933-.071731 1.020174-.231133 .74122-.653549C1.028144-.653549 1.227397-.836862 1.227397-1.099875C1.227397-1.354919 1.044085-1.538232 .789041-1.538232C.573848-1.538232 .350685-1.40274 .350685-1.083935C.350685-.326775 1.163636 .167372 2.072229 .167372C3.132254 .167372 3.873474-.565878 3.873474-1.362889C3.873474-2.024408 3.347447-2.630137 2.534496-2.805479C3.164134-3.028643 3.634371-3.57061 3.634371-4.208219S2.917061-5.300125 2.088169-5.300125C1.235367-5.300125 .589788-4.837858 .589788-4.23213C.589788-3.937235 .789041-3.809714 .996264-3.809714C1.243337-3.809714 1.40274-3.985056 1.40274-4.216189C1.40274-4.511083 1.147696-4.622665 .972354-4.630635C1.307098-5.068991 1.920797-5.092902 2.064259-5.092902C2.271482-5.092902 2.87721-5.029141 2.87721-4.208219C2.87721-3.650311 2.646077-3.315567 2.534496-3.188045C2.295392-2.940971 2.11208-2.925031 1.625903-2.893151C1.474471-2.885181 1.41071-2.87721 1.41071-2.773599C1.41071-2.662017 1.482441-2.662017 1.617933-2.662017H2.016438Z'/> +<path id='g2-56' d='M2.646077-2.87721C3.092403-3.092403 3.634371-3.490909 3.634371-4.112578C3.634371-4.869738 2.86127-5.300125 2.12005-5.300125C1.275218-5.300125 .589788-4.718306 .589788-3.969116C.589788-3.674222 .6934-3.403238 .892653-3.172105C1.028144-3.004732 1.060025-2.988792 1.554172-2.677958C.565878-2.239601 .350685-1.657783 .350685-1.211457C.350685-.334745 1.235367 .167372 2.10411 .167372C3.084433 .167372 3.873474-.494147 3.873474-1.338979C3.873474-1.841096 3.602491-2.175841 3.474969-2.311333C3.339477-2.438854 3.331507-2.446824 2.646077-2.87721ZM1.41071-3.626401C1.179577-3.761893 .988294-3.993026 .988294-4.27198C.988294-4.774097 1.538232-5.092902 2.10411-5.092902C2.725778-5.092902 3.235866-4.670486 3.235866-4.112578C3.235866-3.650311 2.87721-3.259776 2.406974-3.028643L1.41071-3.626401ZM1.801245-2.534496C1.833126-2.518555 2.741719-1.960648 2.87721-1.872976C3.004732-1.801245 3.419178-1.546202 3.419178-1.067995C3.419178-.454296 2.773599-.071731 2.12005-.071731C1.41071-.071731 .804981-.557908 .804981-1.211457C.804981-1.809215 1.251308-2.279452 1.801245-2.534496Z'/> +<path id='g1-18' d='M5.296139-6.01345C5.296139-7.232877 4.913574-8.416438 3.93325-8.416438C2.259527-8.416438 .478207-4.913574 .478207-2.283437C.478207-1.733499 .597758 .119552 1.853051 .119552C3.478954 .119552 5.296139-3.299626 5.296139-6.01345ZM1.673724-4.327771C1.853051-5.033126 2.10411-6.03736 2.582316-6.886177C2.976837-7.603487 3.395268-8.177335 3.921295-8.177335C4.315816-8.177335 4.578829-7.84259 4.578829-6.694894C4.578829-6.264508 4.542964-5.66675 4.196264-4.327771H1.673724ZM4.112578-3.969116C3.813699-2.797509 3.56264-2.044334 3.132254-1.291158C2.785554-.681445 2.367123-.119552 1.865006-.119552C1.494396-.119552 1.195517-.406476 1.195517-1.590037C1.195517-2.367123 1.3868-3.180075 1.578082-3.969116H4.112578Z'/> +<path id='g1-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/> +<path id='g1-70' d='M3.550685-3.897385H4.698381C5.606974-3.897385 5.678705-3.694147 5.678705-3.347447C5.678705-3.19203 5.654795-3.024658 5.595019-2.761644C5.571108-2.713823 5.559153-2.654047 5.559153-2.630137C5.559153-2.546451 5.606974-2.49863 5.69066-2.49863C5.786301-2.49863 5.798257-2.546451 5.846077-2.737733L6.539477-5.523288C6.539477-5.571108 6.503611-5.642839 6.419925-5.642839C6.312329-5.642839 6.300374-5.595019 6.252553-5.391781C6.001494-4.495143 5.762391-4.244085 4.722291-4.244085H3.634371L4.411457-7.340473C4.519054-7.758904 4.542964-7.79477 5.033126-7.79477H6.635118C8.129514-7.79477 8.344707-7.352428 8.344707-6.503611C8.344707-6.43188 8.344707-6.168867 8.308842-5.858032C8.296887-5.810212 8.272976-5.654795 8.272976-5.606974C8.272976-5.511333 8.332752-5.475467 8.404483-5.475467C8.488169-5.475467 8.53599-5.523288 8.5599-5.738481L8.810959-7.830635C8.810959-7.866501 8.834869-7.986052 8.834869-8.009963C8.834869-8.141469 8.727273-8.141469 8.51208-8.141469H2.84533C2.618182-8.141469 2.49863-8.141469 2.49863-7.926276C2.49863-7.79477 2.582316-7.79477 2.785554-7.79477C3.526775-7.79477 3.526775-7.711083 3.526775-7.579577C3.526775-7.519801 3.514819-7.47198 3.478954-7.340473L1.865006-.884682C1.75741-.466252 1.733499-.3467 .896638-.3467C.669489-.3467 .549938-.3467 .549938-.131507C.549938 0 .657534 0 .729265 0C.956413 0 1.195517-.02391 1.422665-.02391H2.976837C3.239851-.02391 3.526775 0 3.789788 0C3.897385 0 4.040847 0 4.040847-.215193C4.040847-.3467 3.969116-.3467 3.706102-.3467C2.761644-.3467 2.737733-.430386 2.737733-.609714C2.737733-.669489 2.761644-.765131 2.785554-.848817L3.550685-3.897385Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.759354)'> +<use x='56.413267' y='63.910285' xlink:href='#g1-18'/> +<use x='61.868442' y='65.703548' xlink:href='#g2-49'/> +<use x='69.921586' y='63.910285' xlink:href='#g3-61'/> +<use x='82.347067' y='63.910285' xlink:href='#g3-52'/> +<use x='88.200057' y='63.910285' xlink:href='#g3-48'/> +<use x='94.053048' y='58.974099' xlink:href='#g0-14'/> +<use x='98.785362' y='63.910285' xlink:href='#g1-70'/> +<use x='56.413267' y='81.344905' xlink:href='#g1-18'/> +<use x='61.868442' y='83.138168' xlink:href='#g2-50'/> +<use x='69.921586' y='81.344905' xlink:href='#g3-61'/> +<use x='82.347067' y='81.344905' xlink:href='#g3-52'/> +<use x='88.200057' y='81.344905' xlink:href='#g3-57'/> +<use x='94.053048' y='76.408719' xlink:href='#g0-14'/> +<use x='98.785362' y='81.344905' xlink:href='#g1-70'/> +<use x='56.413267' y='98.779525' xlink:href='#g1-18'/> +<use x='61.868442' y='100.572788' xlink:href='#g2-51'/> +<use x='69.921586' y='98.779525' xlink:href='#g3-61'/> +<use x='82.347067' y='98.779525' xlink:href='#g3-52'/> +<use x='88.200057' y='98.779525' xlink:href='#g3-53'/> +<use x='94.053048' y='93.843339' xlink:href='#g0-14'/> +<use x='98.785362' y='98.779525' xlink:href='#g1-70'/> +<use x='56.413267' y='116.214145' xlink:href='#g1-58'/> +<use x='59.664928' y='116.214145' xlink:href='#g1-58'/> +<use x='62.91659' y='116.214145' xlink:href='#g1-58'/> +<use x='56.413267' y='133.648765' xlink:href='#g1-18'/> +<use x='61.868442' y='135.442028' xlink:href='#g2-49'/> +<use x='66.102625' y='135.442028' xlink:href='#g2-56'/> +<use x='70.336808' y='135.442028' xlink:href='#g2-48'/> +<use x='78.389952' y='133.648765' xlink:href='#g3-61'/> +<use x='90.815433' y='133.648765' xlink:href='#g3-54'/> +<use x='96.668423' y='133.648765' xlink:href='#g3-48'/> +<use x='102.521413' y='128.712579' xlink:href='#g0-14'/> +<use x='107.253728' y='133.648765' xlink:href='#g1-70'/> +<use x='56.413267' y='151.083385' xlink:href='#g1-18'/> +<use x='61.868442' y='152.876648' xlink:href='#g2-49'/> +<use x='66.102625' y='152.876648' xlink:href='#g2-56'/> +<use x='70.336808' y='152.876648' xlink:href='#g2-49'/> +<use x='78.389952' y='151.083385' xlink:href='#g3-61'/> +<use x='90.815433' y='151.083385' xlink:href='#g3-53'/> +<use x='96.668423' y='151.083385' xlink:href='#g3-54'/> +<use x='102.521413' y='146.147199' xlink:href='#g0-14'/> +<use x='107.253728' y='151.083385' xlink:href='#g1-70'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/ewa-temp2.svg b/_resources/ewa-temp2.svg new file mode 100644 index 0000000..3db4840 --- /dev/null +++ b/_resources/ewa-temp2.svg @@ -0,0 +1,79 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='139.74032pt' height='90.147812pt' viewBox='-.239051 -.236219 139.74032 90.147812'> +<defs> +<path id='g0-0' d='M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z'/> +<path id='g2-116' d='M1.761395-3.172105H2.542466C2.693898-3.172105 2.789539-3.172105 2.789539-3.323537C2.789539-3.435118 2.685928-3.435118 2.550436-3.435118H1.825156L2.11208-4.566874C2.14396-4.686426 2.14396-4.726276 2.14396-4.734247C2.14396-4.901619 2.016438-4.98132 1.880946-4.98132C1.609963-4.98132 1.554172-4.766127 1.466501-4.407472L1.219427-3.435118H.454296C.302864-3.435118 .199253-3.435118 .199253-3.283686C.199253-3.172105 .302864-3.172105 .438356-3.172105H1.155666L.67746-1.259278C.629639-1.060025 .557908-.781071 .557908-.669489C.557908-.191283 .948443 .079701 1.370859 .079701C2.223661 .079701 2.709838-1.044085 2.709838-1.139726C2.709838-1.227397 2.638107-1.243337 2.590286-1.243337C2.502615-1.243337 2.494645-1.211457 2.438854-1.091905C2.279452-.70934 1.880946-.143462 1.39477-.143462C1.227397-.143462 1.131756-.255044 1.131756-.518057C1.131756-.669489 1.155666-.757161 1.179577-.860772L1.761395-3.172105Z'/> +<path id='g1-3' d='M3.287671-5.104857C3.299626-5.272229 3.299626-5.559153 2.988792-5.559153C2.797509-5.559153 2.642092-5.403736 2.677958-5.248319V-5.092902L2.84533-3.239851L1.315068-4.351681C1.207472-4.411457 1.183562-4.435367 1.099875-4.435367C.932503-4.435367 .777086-4.267995 .777086-4.100623C.777086-3.90934 .896638-3.861519 1.016189-3.801743L2.713823-2.988792L1.06401-2.187796C.872727-2.092154 .777086-2.044334 .777086-1.865006S.932503-1.530262 1.099875-1.530262C1.183562-1.530262 1.207472-1.530262 1.506351-1.75741L2.84533-2.725778L2.666002-.71731C2.666002-.466252 2.881196-.406476 2.976837-.406476C3.120299-.406476 3.299626-.490162 3.299626-.71731L3.120299-2.725778L4.65056-1.613948C4.758157-1.554172 4.782067-1.530262 4.865753-1.530262C5.033126-1.530262 5.188543-1.697634 5.188543-1.865006C5.188543-2.044334 5.080946-2.10411 4.937484-2.175841C4.220174-2.534496 4.196264-2.534496 3.251806-2.976837L4.901619-3.777833C5.092902-3.873474 5.188543-3.921295 5.188543-4.100623S5.033126-4.435367 4.865753-4.435367C4.782067-4.435367 4.758157-4.435367 4.459278-4.208219L3.120299-3.239851L3.287671-5.104857Z'/> +<path id='g5-43' d='M4.770112-2.761644H8.069738C8.237111-2.761644 8.452304-2.761644 8.452304-2.976837C8.452304-3.203985 8.249066-3.203985 8.069738-3.203985H4.770112V-6.503611C4.770112-6.670984 4.770112-6.886177 4.554919-6.886177C4.327771-6.886177 4.327771-6.682939 4.327771-6.503611V-3.203985H1.028144C.860772-3.203985 .645579-3.203985 .645579-2.988792C.645579-2.761644 .848817-2.761644 1.028144-2.761644H4.327771V.537983C4.327771 .705355 4.327771 .920548 4.542964 .920548C4.770112 .920548 4.770112 .71731 4.770112 .537983V-2.761644Z'/> +<path id='g5-48' d='M5.355915-3.825654C5.355915-4.817933 5.296139-5.786301 4.865753-6.694894C4.375592-7.687173 3.514819-7.950187 2.929016-7.950187C2.235616-7.950187 1.3868-7.603487 .944458-6.611208C.609714-5.858032 .490162-5.116812 .490162-3.825654C.490162-2.666002 .573848-1.793275 1.004234-.944458C1.470486-.035866 2.295392 .251059 2.917061 .251059C3.957161 .251059 4.554919-.37061 4.901619-1.06401C5.332005-1.960648 5.355915-3.132254 5.355915-3.825654ZM2.917061 .011955C2.534496 .011955 1.75741-.203238 1.530262-1.506351C1.398755-2.223661 1.398755-3.132254 1.398755-3.969116C1.398755-4.94944 1.398755-5.834122 1.590037-6.539477C1.793275-7.340473 2.402989-7.711083 2.917061-7.711083C3.371357-7.711083 4.064757-7.436115 4.291905-6.40797C4.447323-5.726526 4.447323-4.782067 4.447323-3.969116C4.447323-3.16812 4.447323-2.259527 4.315816-1.530262C4.088667-.215193 3.335492 .011955 2.917061 .011955Z'/> +<path id='g5-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +<path id='g5-57' d='M4.375592-3.478954C4.375592-.657534 3.120299-.071731 2.402989-.071731C2.116065-.071731 1.482441-.107597 1.183562-.526027H1.255293C1.338979-.502117 1.769365-.573848 1.769365-1.016189C1.769365-1.279203 1.590037-1.506351 1.279203-1.506351S.777086-1.303113 .777086-.992279C.777086-.251059 1.374844 .251059 2.414944 .251059C3.90934 .251059 5.355915-1.338979 5.355915-3.93325C5.355915-7.149191 4.016936-7.950187 2.964882-7.950187C1.649813-7.950187 .490162-6.850311 .490162-5.272229S1.601993-2.618182 2.797509-2.618182C3.682192-2.618182 4.136488-3.263761 4.375592-3.873474V-3.478954ZM2.84533-2.857285C2.092154-2.857285 1.769365-3.466999 1.661768-3.694147C1.470486-4.148443 1.470486-4.722291 1.470486-5.260274C1.470486-5.929763 1.470486-6.503611 1.78132-6.993773C1.996513-7.316563 2.319303-7.663263 2.964882-7.663263C3.646326-7.663263 3.993026-7.065504 4.112578-6.790535C4.351681-6.204732 4.351681-5.188543 4.351681-5.009215C4.351681-4.004981 3.897385-2.857285 2.84533-2.857285Z'/> +<path id='g5-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g4-48' d='M3.897385-2.542466C3.897385-3.395268 3.809714-3.913325 3.5467-4.423412C3.196015-5.124782 2.550436-5.300125 2.11208-5.300125C1.107846-5.300125 .74122-4.550934 .629639-4.327771C.342715-3.745953 .326775-2.956912 .326775-2.542466C.326775-2.016438 .350685-1.211457 .73325-.573848C1.099875 .01594 1.689664 .167372 2.11208 .167372C2.494645 .167372 3.180075 .047821 3.57858-.74122C3.873474-1.315068 3.897385-2.024408 3.897385-2.542466ZM2.11208-.055791C1.841096-.055791 1.291158-.183313 1.123786-1.020174C1.036115-1.474471 1.036115-2.223661 1.036115-2.638107C1.036115-3.188045 1.036115-3.745953 1.123786-4.184309C1.291158-4.99726 1.912827-5.076961 2.11208-5.076961C2.383064-5.076961 2.933001-4.941469 3.092403-4.216189C3.188045-3.777833 3.188045-3.180075 3.188045-2.638107C3.188045-2.16787 3.188045-1.45056 3.092403-1.004234C2.925031-.167372 2.375093-.055791 2.11208-.055791Z'/> +<path id='g4-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g4-50' d='M2.247572-1.625903C2.375093-1.745455 2.709838-2.008468 2.83736-2.12005C3.331507-2.574346 3.801743-3.012702 3.801743-3.737983C3.801743-4.686426 3.004732-5.300125 2.008468-5.300125C1.052055-5.300125 .422416-4.574844 .422416-3.865504C.422416-3.474969 .73325-3.419178 .844832-3.419178C1.012204-3.419178 1.259278-3.53873 1.259278-3.841594C1.259278-4.25604 .860772-4.25604 .765131-4.25604C.996264-4.837858 1.530262-5.037111 1.920797-5.037111C2.662017-5.037111 3.044583-4.407472 3.044583-3.737983C3.044583-2.909091 2.462765-2.303362 1.522291-1.338979L.518057-.302864C.422416-.215193 .422416-.199253 .422416 0H3.57061L3.801743-1.42665H3.55467C3.53076-1.267248 3.466999-.868742 3.371357-.71731C3.323537-.653549 2.717808-.653549 2.590286-.653549H1.171606L2.247572-1.625903Z'/> +<path id='g3-18' d='M5.296139-6.01345C5.296139-7.232877 4.913574-8.416438 3.93325-8.416438C2.259527-8.416438 .478207-4.913574 .478207-2.283437C.478207-1.733499 .597758 .119552 1.853051 .119552C3.478954 .119552 5.296139-3.299626 5.296139-6.01345ZM1.673724-4.327771C1.853051-5.033126 2.10411-6.03736 2.582316-6.886177C2.976837-7.603487 3.395268-8.177335 3.921295-8.177335C4.315816-8.177335 4.578829-7.84259 4.578829-6.694894C4.578829-6.264508 4.542964-5.66675 4.196264-4.327771H1.673724ZM4.112578-3.969116C3.813699-2.797509 3.56264-2.044334 3.132254-1.291158C2.785554-.681445 2.367123-.119552 1.865006-.119552C1.494396-.119552 1.195517-.406476 1.195517-1.590037C1.195517-2.367123 1.3868-3.180075 1.578082-3.969116H4.112578Z'/> +<path id='g3-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/> +<path id='g3-86' d='M7.400249-6.838356C7.806725-7.483935 8.177335-7.770859 8.787049-7.81868C8.9066-7.830635 9.002242-7.830635 9.002242-8.045828C9.002242-8.093649 8.978331-8.16538 8.870735-8.16538C8.655542-8.16538 8.141469-8.141469 7.926276-8.141469C7.579577-8.141469 7.220922-8.16538 6.886177-8.16538C6.790535-8.16538 6.670984-8.16538 6.670984-7.938232C6.670984-7.830635 6.77858-7.81868 6.826401-7.81868C7.268742-7.782814 7.316563-7.567621 7.316563-7.424159C7.316563-7.244832 7.149191-6.969863 7.137235-6.957908L3.383313-1.004234L2.546451-7.44807C2.546451-7.79477 3.16812-7.81868 3.299626-7.81868C3.478954-7.81868 3.58655-7.81868 3.58655-8.045828C3.58655-8.16538 3.455044-8.16538 3.419178-8.16538C3.21594-8.16538 2.976837-8.141469 2.773599-8.141469H2.10411C1.231382-8.141469 .872727-8.16538 .860772-8.16538C.789041-8.16538 .645579-8.16538 .645579-7.950187C.645579-7.81868 .729265-7.81868 .920548-7.81868C1.530262-7.81868 1.566127-7.711083 1.601993-7.412204L2.558406-.035866C2.594271 .215193 2.594271 .251059 2.761644 .251059C2.905106 .251059 2.964882 .215193 3.084433 .02391L7.400249-6.838356Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -63.227961)'> +<use x='56.413267' y='63.910285' xlink:href='#g3-86'/> +<use x='63.237617' y='65.703548' xlink:href='#g4-48'/> +<use x='71.29075' y='63.910285' xlink:href='#g5-61'/> +<use x='83.716231' y='63.910285' xlink:href='#g5-48'/> +<use x='56.413267' y='81.344905' xlink:href='#g3-86'/> +<use x='63.237617' y='83.138168' xlink:href='#g4-49'/> +<use x='71.29075' y='81.344905' xlink:href='#g5-61'/> +<use x='83.716231' y='81.344905' xlink:href='#g5-48'/> +<use x='89.569221' y='81.344905' xlink:href='#g3-58'/> +<use x='92.820882' y='81.344905' xlink:href='#g5-57'/> +<use x='101.330536' y='81.344905' xlink:href='#g1-3'/> +<use x='109.964806' y='81.344905' xlink:href='#g3-86'/> +<use x='116.789156' y='83.138168' xlink:href='#g4-48'/> +<use x='124.178135' y='81.344905' xlink:href='#g5-43'/> +<use x='135.93945' y='81.344905' xlink:href='#g5-48'/> +<use x='141.79244' y='81.344905' xlink:href='#g3-58'/> +<use x='145.044101' y='81.344905' xlink:href='#g5-49'/> +<use x='153.553755' y='81.344905' xlink:href='#g1-3'/> +<use x='162.188025' y='81.344905' xlink:href='#g3-18'/> +<use x='167.6432' y='83.138168' xlink:href='#g4-49'/> +<use x='56.413267' y='98.779525' xlink:href='#g3-86'/> +<use x='63.237617' y='100.572788' xlink:href='#g4-50'/> +<use x='71.29075' y='98.779525' xlink:href='#g5-61'/> +<use x='83.716231' y='98.779525' xlink:href='#g5-48'/> +<use x='89.569221' y='98.779525' xlink:href='#g3-58'/> +<use x='92.820882' y='98.779525' xlink:href='#g5-57'/> +<use x='101.330536' y='98.779525' xlink:href='#g1-3'/> +<use x='109.964806' y='98.779525' xlink:href='#g3-86'/> +<use x='116.789156' y='100.572788' xlink:href='#g4-49'/> +<use x='124.178135' y='98.779525' xlink:href='#g5-43'/> +<use x='135.93945' y='98.779525' xlink:href='#g5-48'/> +<use x='141.79244' y='98.779525' xlink:href='#g3-58'/> +<use x='145.044101' y='98.779525' xlink:href='#g5-49'/> +<use x='153.553755' y='98.779525' xlink:href='#g1-3'/> +<use x='162.188025' y='98.779525' xlink:href='#g3-18'/> +<use x='167.6432' y='100.572788' xlink:href='#g4-50'/> +<use x='67.969921' y='116.214145' xlink:href='#g3-58'/> +<use x='71.221582' y='116.214145' xlink:href='#g3-58'/> +<use x='74.473243' y='116.214145' xlink:href='#g3-58'/> +<use x='57.589429' y='133.648765' xlink:href='#g3-86'/> +<use x='64.413779' y='135.442028' xlink:href='#g2-116'/> +<use x='71.29075' y='133.648765' xlink:href='#g5-61'/> +<use x='83.716231' y='133.648765' xlink:href='#g5-48'/> +<use x='89.569221' y='133.648765' xlink:href='#g3-58'/> +<use x='92.820882' y='133.648765' xlink:href='#g5-57'/> +<use x='101.330536' y='133.648765' xlink:href='#g1-3'/> +<use x='109.964806' y='133.648765' xlink:href='#g3-86'/> +<use x='116.789156' y='135.442028' xlink:href='#g2-116'/> +<use x='119.847177' y='135.442028' xlink:href='#g0-0'/> +<use x='126.433684' y='135.442028' xlink:href='#g4-49'/> +<use x='133.822662' y='133.648765' xlink:href='#g5-43'/> +<use x='145.583977' y='133.648765' xlink:href='#g5-48'/> +<use x='151.436967' y='133.648765' xlink:href='#g3-58'/> +<use x='154.688629' y='133.648765' xlink:href='#g5-49'/> +<use x='163.198282' y='133.648765' xlink:href='#g1-3'/> +<use x='171.832553' y='133.648765' xlink:href='#g3-18'/> +<use x='177.287728' y='135.442028' xlink:href='#g2-116'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/ewa-weight.svg b/_resources/ewa-weight.svg new file mode 100644 index 0000000..7ddb071 --- /dev/null +++ b/_resources/ewa-weight.svg @@ -0,0 +1,21 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='51.202301pt' height='12.327269pt' viewBox='-.299738 -.256625 51.202301 12.327269'> +<defs> +<path id='g0-0' d='M7.189089-2.50909C7.374543-2.50909 7.570907-2.50909 7.570907-2.727272S7.374543-2.945454 7.189089-2.945454H1.287272C1.101818-2.945454 .905454-2.945454 .905454-2.727272S1.101818-2.50909 1.287272-2.50909H7.189089Z'/> +<path id='g3-40' d='M3.610908 2.618181C3.610908 2.585454 3.610908 2.563636 3.425454 2.378181C2.061818 1.003636 1.712727-1.058182 1.712727-2.727272C1.712727-4.625453 2.127272-6.523635 3.46909-7.887271C3.610908-8.01818 3.610908-8.039998 3.610908-8.072725C3.610908-8.149089 3.567272-8.181816 3.501817-8.181816C3.392726-8.181816 2.410908-7.439998 1.767272-6.054544C1.210909-4.854544 1.08-3.643635 1.08-2.727272C1.08-1.876363 1.2-.556363 1.799999 .676363C2.454545 2.018181 3.392726 2.727272 3.501817 2.727272C3.567272 2.727272 3.610908 2.694545 3.610908 2.618181Z'/> +<path id='g3-41' d='M3.152726-2.727272C3.152726-3.578181 3.032726-4.89818 2.432727-6.130907C1.778181-7.472725 .84-8.181816 .730909-8.181816C.665454-8.181816 .621818-8.13818 .621818-8.072725C.621818-8.039998 .621818-8.01818 .829091-7.821816C1.898181-6.741816 2.519999-5.007271 2.519999-2.727272C2.519999-.861818 2.116363 1.058182 .763636 2.432727C.621818 2.563636 .621818 2.585454 .621818 2.618181C.621818 2.683636 .665454 2.727272 .730909 2.727272C.84 2.727272 1.821818 1.985454 2.465454 .6C3.021817-.6 3.152726-1.810909 3.152726-2.727272Z'/> +<path id='g3-49' d='M3.207272-6.981816C3.207272-7.243634 3.207272-7.265453 2.956363-7.265453C2.279999-6.567271 1.32-6.567271 .970909-6.567271V-6.229089C1.189091-6.229089 1.832727-6.229089 2.399999-6.512725V-.861818C2.399999-.469091 2.367272-.338182 1.385454-.338182H1.036363V0C1.418181-.032727 2.367272-.032727 2.803636-.032727S4.18909-.032727 4.570908 0V-.338182H4.221817C3.239999-.338182 3.207272-.458182 3.207272-.861818V-6.981816Z'/> +<path id='g1-110' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.833126-2.279452 1.833126-2.287422 2.016438-2.550436C2.279452-2.940971 2.654047-3.291656 3.188045-3.291656C3.474969-3.291656 3.642341-3.124284 3.642341-2.749689C3.642341-2.311333 3.307597-1.40274 3.156164-1.012204C3.052553-.749191 3.052553-.70137 3.052553-.597758C3.052553-.143462 3.427148 .079701 3.769863 .079701C4.550934 .079701 4.877709-1.036115 4.877709-1.139726C4.877709-1.219427 4.813948-1.243337 4.758157-1.243337C4.662516-1.243337 4.646575-1.187547 4.622665-1.107846C4.431382-.454296 4.096638-.143462 3.793773-.143462C3.666252-.143462 3.602491-.223163 3.602491-.406476S3.666252-.765131 3.745953-.964384C3.865504-1.267248 4.216189-2.183811 4.216189-2.630137C4.216189-3.227895 3.801743-3.514819 3.227895-3.514819C2.582316-3.514819 2.16787-3.124284 1.936737-2.82142C1.880946-3.259776 1.530262-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.318804-2.709838 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g2-12' d='M6.272726-6.207271C6.272726-7.025453 5.672726-7.701816 4.78909-7.701816C4.156362-7.701816 3.850908-7.527271 3.46909-7.243634C2.86909-6.807271 2.26909-5.749089 2.061818-4.919999L.327273 2.007272C.316364 2.050909 .370909 2.116363 .458182 2.116363S.578182 2.094545 .589091 2.061818L1.352727-.949091C1.56-.294545 2.039999 .109091 2.836363 .109091S4.450908-.272727 4.941817-.752727C5.465453-1.254545 5.814544-1.952727 5.814544-2.759999C5.814544-3.545454 5.410908-4.112726 5.01818-4.385453C5.650908-4.745453 6.272726-5.421817 6.272726-6.207271ZM4.30909-4.396362C4.167272-4.341817 4.047272-4.319999 3.785453-4.319999C3.632726-4.319999 3.414545-4.30909 3.305454-4.363635C3.327272-4.472726 3.719999-4.450908 3.839999-4.450908C4.06909-4.450908 4.167272-4.450908 4.30909-4.396362ZM5.661817-6.45818C5.661817-5.694544 5.247271-4.90909 4.679999-4.570908C4.385453-4.679999 4.167272-4.701817 3.839999-4.701817C3.610908-4.701817 2.999999-4.712726 2.999999-4.352726C2.98909-4.047272 3.556363-4.079999 3.752726-4.079999C4.156362-4.079999 4.319999-4.090908 4.647271-4.221817C5.061817-3.82909 5.116362-3.490908 5.127271-2.98909C5.149089-2.356363 4.887271-1.538181 4.581817-1.112727C4.156362-.523636 3.425454-.130909 2.803636-.130909C1.974545-.130909 1.56-.763636 1.56-1.538181C1.56-1.647272 1.56-1.810909 1.614545-2.018181L2.312727-4.77818C2.552727-5.716362 3.338181-7.450907 4.647271-7.450907C5.279999-7.450907 5.661817-7.112725 5.661817-6.45818Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -68.689878)'> +<use x='70.734745' y='68.742217' xlink:href='#g2-12'/> +<use x='77.480962' y='64.783399' xlink:href='#g1-110'/> +<use x='83.117297' y='68.742217' xlink:href='#g3-40'/> +<use x='87.359735' y='68.742217' xlink:href='#g3-49'/> +<use x='95.238497' y='68.742217' xlink:href='#g0-0'/> +<use x='106.147571' y='68.742217' xlink:href='#g2-12'/> +<use x='112.893789' y='68.742217' xlink:href='#g3-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/ewa.svg b/_resources/ewa.svg new file mode 100644 index 0000000..d6b34f3 --- /dev/null +++ b/_resources/ewa.svg @@ -0,0 +1,123 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="224.833863pt" height="98.801049pt" viewBox="-.239051 -.232825 224.833863 98.801049"> +<defs> +<path id="g0-88" d="M15.135243 16.737235L16.581818 12.911582H16.282939C15.816687 14.154919 14.54944 14.96787 13.174595 15.326526C12.923537 15.386301 11.75193 15.697136 9.456538 15.697136H2.247572L8.332752 8.5599C8.416438 8.464259 8.440349 8.428394 8.440349 8.368618C8.440349 8.344707 8.440349 8.308842 8.356663 8.18929L2.785554 .573848H9.336986C10.938979 .573848 12.026899 .74122 12.134496 .765131C12.780075 .860772 13.820174 1.06401 14.764633 1.661768C15.063512 1.853051 15.876463 2.391034 16.282939 3.359402H16.581818L15.135243 0H1.004234C.729265 0 .71731 .011955 .681445 .083686C.669489 .119552 .669489 .3467 .669489 .478207L6.993773 9.133748L.800996 16.390535C.681445 16.533998 .681445 16.593773 .681445 16.605729C.681445 16.737235 .789041 16.737235 1.004234 16.737235H15.135243Z"/> +<path id="g2-0" d="M7.878456-2.749689C8.081694-2.749689 8.296887-2.749689 8.296887-2.988792S8.081694-3.227895 7.878456-3.227895H1.41071C1.207472-3.227895 .992279-3.227895 .992279-2.988792S1.207472-2.749689 1.41071-2.749689H7.878456Z"/> +<path id="g5-48" d="M3.897385-2.542466C3.897385-3.395268 3.809714-3.913325 3.5467-4.423412C3.196015-5.124782 2.550436-5.300125 2.11208-5.300125C1.107846-5.300125 .74122-4.550934 .629639-4.327771C.342715-3.745953 .326775-2.956912 .326775-2.542466C.326775-2.016438 .350685-1.211457 .73325-.573848C1.099875 .01594 1.689664 .167372 2.11208 .167372C2.494645 .167372 3.180075 .047821 3.57858-.74122C3.873474-1.315068 3.897385-2.024408 3.897385-2.542466ZM2.11208-.055791C1.841096-.055791 1.291158-.183313 1.123786-1.020174C1.036115-1.474471 1.036115-2.223661 1.036115-2.638107C1.036115-3.188045 1.036115-3.745953 1.123786-4.184309C1.291158-4.99726 1.912827-5.076961 2.11208-5.076961C2.383064-5.076961 2.933001-4.941469 3.092403-4.216189C3.188045-3.777833 3.188045-3.180075 3.188045-2.638107C3.188045-2.16787 3.188045-1.45056 3.092403-1.004234C2.925031-.167372 2.375093-.055791 2.11208-.055791Z"/> +<path id="g5-49" d="M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z"/> +<path id="g5-50" d="M2.247572-1.625903C2.375093-1.745455 2.709838-2.008468 2.83736-2.12005C3.331507-2.574346 3.801743-3.012702 3.801743-3.737983C3.801743-4.686426 3.004732-5.300125 2.008468-5.300125C1.052055-5.300125 .422416-4.574844 .422416-3.865504C.422416-3.474969 .73325-3.419178 .844832-3.419178C1.012204-3.419178 1.259278-3.53873 1.259278-3.841594C1.259278-4.25604 .860772-4.25604 .765131-4.25604C.996264-4.837858 1.530262-5.037111 1.920797-5.037111C2.662017-5.037111 3.044583-4.407472 3.044583-3.737983C3.044583-2.909091 2.462765-2.303362 1.522291-1.338979L.518057-.302864C.422416-.215193 .422416-.199253 .422416 0H3.57061L3.801743-1.42665H3.55467C3.53076-1.267248 3.466999-.868742 3.371357-.71731C3.323537-.653549 2.717808-.653549 2.590286-.653549H1.171606L2.247572-1.625903Z"/> +<path id="g5-61" d="M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z"/> +<path id="g1-0" d="M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z"/> +<path id="g6-40" d="M3.88543 2.905106C3.88543 2.86924 3.88543 2.84533 3.682192 2.642092C2.486675 1.43462 1.817186-.537983 1.817186-2.976837C1.817186-5.296139 2.379078-7.292653 3.765878-8.703362C3.88543-8.810959 3.88543-8.834869 3.88543-8.870735C3.88543-8.942466 3.825654-8.966376 3.777833-8.966376C3.622416-8.966376 2.642092-8.105604 2.056289-6.933998C1.446575-5.726526 1.171606-4.447323 1.171606-2.976837C1.171606-1.912827 1.338979-.490162 1.960648 .789041C2.666002 2.223661 3.646326 3.000747 3.777833 3.000747C3.825654 3.000747 3.88543 2.976837 3.88543 2.905106Z"/> +<path id="g6-41" d="M3.371357-2.976837C3.371357-3.88543 3.251806-5.36787 2.582316-6.75467C1.876961-8.18929 .896638-8.966376 .765131-8.966376C.71731-8.966376 .657534-8.942466 .657534-8.870735C.657534-8.834869 .657534-8.810959 .860772-8.607721C2.056289-7.400249 2.725778-5.427646 2.725778-2.988792C2.725778-.669489 2.163885 1.327024 .777086 2.737733C.657534 2.84533 .657534 2.86924 .657534 2.905106C.657534 2.976837 .71731 3.000747 .765131 3.000747C.920548 3.000747 1.900872 2.139975 2.486675 .968369C3.096389-.251059 3.371357-1.542217 3.371357-2.976837Z"/> +<path id="g6-43" d="M4.770112-2.761644H8.069738C8.237111-2.761644 8.452304-2.761644 8.452304-2.976837C8.452304-3.203985 8.249066-3.203985 8.069738-3.203985H4.770112V-6.503611C4.770112-6.670984 4.770112-6.886177 4.554919-6.886177C4.327771-6.886177 4.327771-6.682939 4.327771-6.503611V-3.203985H1.028144C.860772-3.203985 .645579-3.203985 .645579-2.988792C.645579-2.761644 .848817-2.761644 1.028144-2.761644H4.327771V.537983C4.327771 .705355 4.327771 .920548 4.542964 .920548C4.770112 .920548 4.770112 .71731 4.770112 .537983V-2.761644Z"/> +<path id="g6-49" d="M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z"/> +<path id="g6-61" d="M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z"/> +<path id="g6-91" d="M2.988792 2.988792V2.546451H1.829141V-8.524035H2.988792V-8.966376H1.3868V2.988792H2.988792Z"/> +<path id="g6-93" d="M1.853051-8.966376H.251059V-8.524035H1.41071V2.546451H.251059V2.988792H1.853051V-8.966376Z"/> +<path id="g3-105" d="M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z"/> +<path id="g3-110" d="M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.833126-2.279452 1.833126-2.287422 2.016438-2.550436C2.279452-2.940971 2.654047-3.291656 3.188045-3.291656C3.474969-3.291656 3.642341-3.124284 3.642341-2.749689C3.642341-2.311333 3.307597-1.40274 3.156164-1.012204C3.052553-.749191 3.052553-.70137 3.052553-.597758C3.052553-.143462 3.427148 .079701 3.769863 .079701C4.550934 .079701 4.877709-1.036115 4.877709-1.139726C4.877709-1.219427 4.813948-1.243337 4.758157-1.243337C4.662516-1.243337 4.646575-1.187547 4.622665-1.107846C4.431382-.454296 4.096638-.143462 3.793773-.143462C3.666252-.143462 3.602491-.223163 3.602491-.406476S3.666252-.765131 3.745953-.964384C3.865504-1.267248 4.216189-2.183811 4.216189-2.630137C4.216189-3.227895 3.801743-3.514819 3.227895-3.514819C2.582316-3.514819 2.16787-3.124284 1.936737-2.82142C1.880946-3.259776 1.530262-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.318804-2.709838 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z"/> +<path id="g3-116" d="M1.761395-3.172105H2.542466C2.693898-3.172105 2.789539-3.172105 2.789539-3.323537C2.789539-3.435118 2.685928-3.435118 2.550436-3.435118H1.825156L2.11208-4.566874C2.14396-4.686426 2.14396-4.726276 2.14396-4.734247C2.14396-4.901619 2.016438-4.98132 1.880946-4.98132C1.609963-4.98132 1.554172-4.766127 1.466501-4.407472L1.219427-3.435118H.454296C.302864-3.435118 .199253-3.435118 .199253-3.283686C.199253-3.172105 .302864-3.172105 .438356-3.172105H1.155666L.67746-1.259278C.629639-1.060025 .557908-.781071 .557908-.669489C.557908-.191283 .948443 .079701 1.370859 .079701C2.223661 .079701 2.709838-1.044085 2.709838-1.139726C2.709838-1.227397 2.638107-1.243337 2.590286-1.243337C2.502615-1.243337 2.494645-1.211457 2.438854-1.091905C2.279452-.70934 1.880946-.143462 1.39477-.143462C1.227397-.143462 1.131756-.255044 1.131756-.518057C1.131756-.669489 1.155666-.757161 1.179577-.860772L1.761395-3.172105Z"/> +<path id="g4-12" d="M6.766625-6.957908C6.766625-7.675218 6.156912-8.428394 5.068991-8.428394C3.526775-8.428394 2.546451-6.539477 2.235616-5.296139L.3467 2.199751C.32279 2.295392 .394521 2.319303 .454296 2.319303C.537983 2.319303 .597758 2.307347 .609714 2.247572L1.446575-1.099875C1.566127-.430386 2.223661 .119552 2.929016 .119552C4.638605 .119552 6.252553-1.219427 6.252553-3.000747C6.252553-3.455044 6.144956-3.90934 5.893898-4.291905C5.750436-4.519054 5.571108-4.686426 5.379826-4.829888C6.240598-5.284184 6.766625-6.01345 6.766625-6.957908ZM4.686426-4.841843C4.495143-4.770112 4.303861-4.746202 4.076712-4.746202C3.90934-4.746202 3.753923-4.734247 3.53873-4.805978C3.658281-4.889664 3.837609-4.913574 4.088667-4.913574C4.303861-4.913574 4.519054-4.889664 4.686426-4.841843ZM6.144956-7.065504C6.144956-6.40797 5.822167-5.451557 5.045081-5.009215C4.817933-5.092902 4.507098-5.152677 4.244085-5.152677C3.993026-5.152677 3.275716-5.176588 3.275716-4.794022C3.275716-4.471233 3.93325-4.507098 4.136488-4.507098C4.447323-4.507098 4.722291-4.578829 5.009215-4.662516C5.391781-4.351681 5.559153-3.945205 5.559153-3.347447C5.559153-2.654047 5.36787-2.092154 5.140722-1.578082C4.746202-.6934 3.813699-.119552 2.988792-.119552C2.116065-.119552 1.661768-.812951 1.661768-1.625903C1.661768-1.733499 1.661768-1.888917 1.709589-2.068244L2.486675-5.212453C2.881196-6.77858 3.88543-8.18929 5.045081-8.18929C5.905853-8.18929 6.144956-7.591532 6.144956-7.065504Z"/> +<path id="g4-18" d="M5.296139-6.01345C5.296139-7.232877 4.913574-8.416438 3.93325-8.416438C2.259527-8.416438 .478207-4.913574 .478207-2.283437C.478207-1.733499 .597758 .119552 1.853051 .119552C3.478954 .119552 5.296139-3.299626 5.296139-6.01345ZM1.673724-4.327771C1.853051-5.033126 2.10411-6.03736 2.582316-6.886177C2.976837-7.603487 3.395268-8.177335 3.921295-8.177335C4.315816-8.177335 4.578829-7.84259 4.578829-6.694894C4.578829-6.264508 4.542964-5.66675 4.196264-4.327771H1.673724ZM4.112578-3.969116C3.813699-2.797509 3.56264-2.044334 3.132254-1.291158C2.785554-.681445 2.367123-.119552 1.865006-.119552C1.494396-.119552 1.195517-.406476 1.195517-1.590037C1.195517-2.367123 1.3868-3.180075 1.578082-3.969116H4.112578Z"/> +<path id="g4-86" d="M7.400249-6.838356C7.806725-7.483935 8.177335-7.770859 8.787049-7.81868C8.9066-7.830635 9.002242-7.830635 9.002242-8.045828C9.002242-8.093649 8.978331-8.16538 8.870735-8.16538C8.655542-8.16538 8.141469-8.141469 7.926276-8.141469C7.579577-8.141469 7.220922-8.16538 6.886177-8.16538C6.790535-8.16538 6.670984-8.16538 6.670984-7.938232C6.670984-7.830635 6.77858-7.81868 6.826401-7.81868C7.268742-7.782814 7.316563-7.567621 7.316563-7.424159C7.316563-7.244832 7.149191-6.969863 7.137235-6.957908L3.383313-1.004234L2.546451-7.44807C2.546451-7.79477 3.16812-7.81868 3.299626-7.81868C3.478954-7.81868 3.58655-7.81868 3.58655-8.045828C3.58655-8.16538 3.455044-8.16538 3.419178-8.16538C3.21594-8.16538 2.976837-8.141469 2.773599-8.141469H2.10411C1.231382-8.141469 .872727-8.16538 .860772-8.16538C.789041-8.16538 .645579-8.16538 .645579-7.950187C.645579-7.81868 .729265-7.81868 .920548-7.81868C1.530262-7.81868 1.566127-7.711083 1.601993-7.412204L2.558406-.035866C2.594271 .215193 2.594271 .251059 2.761644 .251059C2.905106 .251059 2.964882 .215193 3.084433 .02391L7.400249-6.838356Z"/> +</defs> +<g id="page1" transform="matrix(1.13 0 0 1.13 -63.986043 -62.319441)"> +<use x="56.413267" y="63.910285" xlink:href="#g4-86"/> +<use x="63.237617" y="65.703548" xlink:href="#g3-116"/> +<use x="70.114588" y="63.910285" xlink:href="#g6-61"/> +<use x="82.540069" y="63.910285" xlink:href="#g4-12"/> +<use x="89.81134" y="63.910285" xlink:href="#g4-86"/> +<use x="96.63569" y="65.703548" xlink:href="#g3-116"/> +<use x="99.693711" y="65.703548" xlink:href="#g1-0"/> +<use x="106.280217" y="65.703548" xlink:href="#g5-49"/> +<use x="113.669196" y="63.910285" xlink:href="#g6-43"/> +<use x="125.430511" y="63.910285" xlink:href="#g6-40"/> +<use x="129.982836" y="63.910285" xlink:href="#g6-49"/> +<use x="138.49249" y="63.910285" xlink:href="#g2-0"/> +<use x="150.447651" y="63.910285" xlink:href="#g4-12"/> +<use x="157.718922" y="63.910285" xlink:href="#g6-41"/> +<use x="162.271247" y="63.910285" xlink:href="#g4-18"/> +<use x="167.726422" y="65.703548" xlink:href="#g3-116"/> +<use x="70.114588" y="81.344905" xlink:href="#g6-61"/> +<use x="82.540069" y="81.344905" xlink:href="#g4-12"/> +<use x="89.81134" y="81.344905" xlink:href="#g6-91"/> +<use x="93.063001" y="81.344905" xlink:href="#g4-12"/> +<use x="100.334272" y="81.344905" xlink:href="#g4-86"/> +<use x="107.158622" y="83.138168" xlink:href="#g3-116"/> +<use x="110.216643" y="83.138168" xlink:href="#g1-0"/> +<use x="116.80315" y="83.138168" xlink:href="#g5-50"/> +<use x="124.192128" y="81.344905" xlink:href="#g6-43"/> +<use x="135.953443" y="81.344905" xlink:href="#g6-40"/> +<use x="140.505769" y="81.344905" xlink:href="#g6-49"/> +<use x="149.015422" y="81.344905" xlink:href="#g2-0"/> +<use x="160.970583" y="81.344905" xlink:href="#g4-12"/> +<use x="168.241854" y="81.344905" xlink:href="#g6-41"/> +<use x="172.79418" y="81.344905" xlink:href="#g4-18"/> +<use x="178.249354" y="83.138168" xlink:href="#g3-116"/> +<use x="181.307375" y="83.138168" xlink:href="#g1-0"/> +<use x="187.893882" y="83.138168" xlink:href="#g5-49"/> +<use x="192.626197" y="81.344905" xlink:href="#g6-93"/> +<use x="198.534522" y="81.344905" xlink:href="#g6-43"/> +<use x="210.295837" y="81.344905" xlink:href="#g6-40"/> +<use x="214.848162" y="81.344905" xlink:href="#g6-49"/> +<use x="223.357816" y="81.344905" xlink:href="#g2-0"/> +<use x="235.312977" y="81.344905" xlink:href="#g4-12"/> +<use x="242.584248" y="81.344905" xlink:href="#g6-41"/> +<use x="247.136573" y="81.344905" xlink:href="#g4-18"/> +<use x="252.591748" y="83.138168" xlink:href="#g3-116"/> +<use x="70.114588" y="98.779525" xlink:href="#g6-61"/> +<use x="82.540069" y="98.779525" xlink:href="#g4-12"/> +<use x="89.81134" y="93.843339" xlink:href="#g5-50"/> +<use x="94.543655" y="98.779525" xlink:href="#g4-86"/> +<use x="101.368005" y="100.572788" xlink:href="#g3-116"/> +<use x="104.426026" y="100.572788" xlink:href="#g1-0"/> +<use x="111.012532" y="100.572788" xlink:href="#g5-50"/> +<use x="118.401511" y="98.779525" xlink:href="#g6-43"/> +<use x="130.162826" y="98.779525" xlink:href="#g4-12"/> +<use x="137.434096" y="98.779525" xlink:href="#g6-40"/> +<use x="141.986422" y="98.779525" xlink:href="#g6-49"/> +<use x="150.496076" y="98.779525" xlink:href="#g2-0"/> +<use x="162.451236" y="98.779525" xlink:href="#g4-12"/> +<use x="169.722507" y="98.779525" xlink:href="#g6-41"/> +<use x="174.274833" y="98.779525" xlink:href="#g4-18"/> +<use x="179.730008" y="100.572788" xlink:href="#g3-116"/> +<use x="182.788029" y="100.572788" xlink:href="#g1-0"/> +<use x="189.374536" y="100.572788" xlink:href="#g5-49"/> +<use x="196.763514" y="98.779525" xlink:href="#g6-43"/> +<use x="208.524829" y="98.779525" xlink:href="#g6-40"/> +<use x="213.077155" y="98.779525" xlink:href="#g6-49"/> +<use x="221.586808" y="98.779525" xlink:href="#g2-0"/> +<use x="233.541969" y="98.779525" xlink:href="#g4-12"/> +<use x="240.81324" y="98.779525" xlink:href="#g6-41"/> +<use x="245.365566" y="98.779525" xlink:href="#g4-18"/> +<use x="250.82074" y="100.572788" xlink:href="#g3-116"/> +<use x="70.114588" y="128.374203" xlink:href="#g6-61"/> +<use x="82.540069" y="128.374203" xlink:href="#g4-12"/> +<use x="89.81134" y="123.438017" xlink:href="#g3-110"/> +<use x="95.447674" y="128.374203" xlink:href="#g4-86"/> +<use x="102.272024" y="130.167466" xlink:href="#g3-116"/> +<use x="105.330045" y="130.167466" xlink:href="#g1-0"/> +<use x="111.916552" y="130.167466" xlink:href="#g3-110"/> +<use x="120.20955" y="128.374203" xlink:href="#g6-43"/> +<use x="131.970865" y="128.374203" xlink:href="#g6-40"/> +<use x="136.52319" y="128.374203" xlink:href="#g6-49"/> +<use x="145.032844" y="128.374203" xlink:href="#g2-0"/> +<use x="156.988005" y="128.374203" xlink:href="#g4-12"/> +<use x="164.259276" y="128.374203" xlink:href="#g6-41"/> +<use x="171.458961" y="113.430182" xlink:href="#g3-110"/> +<use x="176.597164" y="113.430182" xlink:href="#g1-0"/> +<use x="183.183671" y="113.430182" xlink:href="#g5-49"/> +<use x="170.804099" y="117.016738" xlink:href="#g0-88"/> +<use x="172.586493" y="142.211093" xlink:href="#g3-105"/> +<use x="175.469632" y="142.211093" xlink:href="#g5-61"/> +<use x="182.056139" y="142.211093" xlink:href="#g5-48"/> +<use x="190.065213" y="128.374203" xlink:href="#g4-12"/> +<use x="197.336484" y="123.438017" xlink:href="#g3-105"/> +<use x="200.717756" y="128.374203" xlink:href="#g4-18"/> +<use x="206.172931" y="130.167466" xlink:href="#g3-116"/> +<use x="209.230952" y="130.167466" xlink:href="#g1-0"/> +<use x="215.817458" y="130.167466" xlink:href="#g3-105"/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/f2a731b0ccf8dc8c852572318d9d6bea.png b/_resources/f2a731b0ccf8dc8c852572318d9d6bea.png new file mode 100644 index 0000000..2872394 Binary files /dev/null and b/_resources/f2a731b0ccf8dc8c852572318d9d6bea.png differ diff --git a/_resources/f5d936aaf30e3634df936c86cd99003e.png b/_resources/f5d936aaf30e3634df936c86cd99003e.png new file mode 100644 index 0000000..6aab5ff Binary files /dev/null and b/_resources/f5d936aaf30e3634df936c86cd99003e.png differ diff --git a/_resources/f7e03cc71bfb39115e9ccb74f9306f2e.png b/_resources/f7e03cc71bfb39115e9ccb74f9306f2e.png new file mode 100644 index 0000000..af6359a Binary files /dev/null and b/_resources/f7e03cc71bfb39115e9ccb74f9306f2e.png differ diff --git a/_resources/f96c721f766385e5fc346faf159dcd20.png b/_resources/f96c721f766385e5fc346faf159dcd20.png new file mode 100644 index 0000000..1afb082 Binary files /dev/null and b/_resources/f96c721f766385e5fc346faf159dcd20.png differ diff --git a/_resources/face-recognition.png b/_resources/face-recognition.png new file mode 100644 index 0000000..8b97b0b Binary files /dev/null and b/_resources/face-recognition.png differ diff --git a/_resources/fe7d21e251098349149692c251f53dbc.png b/_resources/fe7d21e251098349149692c251f53dbc.png new file mode 100644 index 0000000..d5fa070 Binary files /dev/null and b/_resources/fe7d21e251098349149692c251f53dbc.png differ diff --git a/_resources/glove-model.svg b/_resources/glove-model.svg new file mode 100644 index 0000000..5fa09fb --- /dev/null +++ b/_resources/glove-model.svg @@ -0,0 +1,90 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='317.302253pt' height='17.503834pt' viewBox='-.299738 -.246346 317.302253 17.503834'> +<defs> +<path id='g2-0' d='M7.189089-2.50909C7.374543-2.50909 7.570907-2.50909 7.570907-2.727272S7.374543-2.945454 7.189089-2.945454H1.287272C1.101818-2.945454 .905454-2.945454 .905454-2.727272S1.101818-2.50909 1.287272-2.50909H7.189089Z'/> +<path id='g1-48' d='M2.11208-3.777833C2.15193-3.881445 2.183811-3.937235 2.183811-4.016936C2.183811-4.27995 1.944707-4.455293 1.721544-4.455293C1.40274-4.455293 1.315068-4.176339 1.283188-4.064757L.270984-.629639C.239103-.533998 .239103-.510087 .239103-.502117C.239103-.430386 .286924-.414446 .366625-.390535C.510087-.326775 .526027-.326775 .541968-.326775C.565878-.326775 .613699-.326775 .669489-.462267L2.11208-3.777833Z'/> +<path id='g6-40' d='M3.610908 2.618181C3.610908 2.585454 3.610908 2.563636 3.425454 2.378181C2.061818 1.003636 1.712727-1.058182 1.712727-2.727272C1.712727-4.625453 2.127272-6.523635 3.46909-7.887271C3.610908-8.01818 3.610908-8.039998 3.610908-8.072725C3.610908-8.149089 3.567272-8.181816 3.501817-8.181816C3.392726-8.181816 2.410908-7.439998 1.767272-6.054544C1.210909-4.854544 1.08-3.643635 1.08-2.727272C1.08-1.876363 1.2-.556363 1.799999 .676363C2.454545 2.018181 3.392726 2.727272 3.501817 2.727272C3.567272 2.727272 3.610908 2.694545 3.610908 2.618181Z'/> +<path id='g6-41' d='M3.152726-2.727272C3.152726-3.578181 3.032726-4.89818 2.432727-6.130907C1.778181-7.472725 .84-8.181816 .730909-8.181816C.665454-8.181816 .621818-8.13818 .621818-8.072725C.621818-8.039998 .621818-8.01818 .829091-7.821816C1.898181-6.741816 2.519999-5.007271 2.519999-2.727272C2.519999-.861818 2.116363 1.058182 .763636 2.432727C.621818 2.563636 .621818 2.585454 .621818 2.618181C.621818 2.683636 .665454 2.727272 .730909 2.727272C.84 2.727272 1.821818 1.985454 2.465454 .6C3.021817-.6 3.152726-1.810909 3.152726-2.727272Z'/> +<path id='g6-43' d='M4.461817-2.50909H7.505452C7.65818-2.50909 7.865452-2.50909 7.865452-2.727272S7.65818-2.945454 7.505452-2.945454H4.461817V-5.999998C4.461817-6.152726 4.461817-6.359998 4.243635-6.359998S4.025453-6.152726 4.025453-5.999998V-2.945454H.970909C.818182-2.945454 .610909-2.945454 .610909-2.727272S.818182-2.50909 .970909-2.50909H4.025453V.545454C4.025453 .698182 4.025453 .905454 4.243635 .905454S4.461817 .698182 4.461817 .545454V-2.50909Z'/> +<path id='g5-48' d='M3.897385-2.542466C3.897385-3.395268 3.809714-3.913325 3.5467-4.423412C3.196015-5.124782 2.550436-5.300125 2.11208-5.300125C1.107846-5.300125 .74122-4.550934 .629639-4.327771C.342715-3.745953 .326775-2.956912 .326775-2.542466C.326775-2.016438 .350685-1.211457 .73325-.573848C1.099875 .01594 1.689664 .167372 2.11208 .167372C2.494645 .167372 3.180075 .047821 3.57858-.74122C3.873474-1.315068 3.897385-2.024408 3.897385-2.542466ZM2.11208-.055791C1.841096-.055791 1.291158-.183313 1.123786-1.020174C1.036115-1.474471 1.036115-2.223661 1.036115-2.638107C1.036115-3.188045 1.036115-3.745953 1.123786-4.184309C1.291158-4.99726 1.912827-5.076961 2.11208-5.076961C2.383064-5.076961 2.933001-4.941469 3.092403-4.216189C3.188045-3.777833 3.188045-3.180075 3.188045-2.638107C3.188045-2.16787 3.188045-1.45056 3.092403-1.004234C2.925031-.167372 2.375093-.055791 2.11208-.055791Z'/> +<path id='g5-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g5-50' d='M2.247572-1.625903C2.375093-1.745455 2.709838-2.008468 2.83736-2.12005C3.331507-2.574346 3.801743-3.012702 3.801743-3.737983C3.801743-4.686426 3.004732-5.300125 2.008468-5.300125C1.052055-5.300125 .422416-4.574844 .422416-3.865504C.422416-3.474969 .73325-3.419178 .844832-3.419178C1.012204-3.419178 1.259278-3.53873 1.259278-3.841594C1.259278-4.25604 .860772-4.25604 .765131-4.25604C.996264-4.837858 1.530262-5.037111 1.920797-5.037111C2.662017-5.037111 3.044583-4.407472 3.044583-3.737983C3.044583-2.909091 2.462765-2.303362 1.522291-1.338979L.518057-.302864C.422416-.215193 .422416-.199253 .422416 0H3.57061L3.801743-1.42665H3.55467C3.53076-1.267248 3.466999-.868742 3.371357-.71731C3.323537-.653549 2.717808-.653549 2.590286-.653549H1.171606L2.247572-1.625903Z'/> +<path id='g5-61' d='M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z'/> +<path id='g0-80' d='M4.592726 5.825453L.72 10.614542C.632727 10.723633 .621818 10.745452 .621818 10.789088C.621818 10.909088 .72 10.909088 .916363 10.909088H9.959997L10.898179 8.192725H10.625452C10.352724 9.010907 9.621816 9.676361 8.694543 9.992724C8.519998 10.04727 7.767271 10.309088 6.163635 10.309088H1.527272L5.312726 5.629089C5.389089 5.530908 5.410908 5.49818 5.410908 5.454544S5.399998 5.399998 5.334544 5.301817L1.78909 .436364H6.109089C7.352725 .436364 9.861815 .512727 10.625452 2.552727H10.898179L9.959997 0H.916363C.621818 0 .610909 .010909 .610909 .349091L4.592726 5.825453Z'/> +<path id='g3-59' d='M1.490411-.119552C1.490411 .398506 1.378829 .852802 .884682 1.346949C.852802 1.370859 .836862 1.3868 .836862 1.42665C.836862 1.490411 .900623 1.538232 .956413 1.538232C1.052055 1.538232 1.713574 .908593 1.713574-.02391C1.713574-.533998 1.522291-.884682 1.171606-.884682C.892653-.884682 .73325-.661519 .73325-.446326C.73325-.223163 .884682 0 1.179577 0C1.370859 0 1.490411-.111582 1.490411-.119552Z'/> +<path id='g3-84' d='M3.602491-4.821918C3.674222-5.108842 3.682192-5.124782 4.008966-5.124782H4.614695C5.443587-5.124782 5.539228-4.861768 5.539228-4.463263C5.539228-4.26401 5.491407-3.921295 5.483437-3.881445C5.467497-3.793773 5.459527-3.722042 5.459527-3.706102C5.459527-3.602491 5.531258-3.57858 5.579078-3.57858C5.66675-3.57858 5.69863-3.626401 5.72254-3.777833L5.937733-5.276214C5.937733-5.387796 5.842092-5.387796 5.69863-5.387796H1.004234C.804981-5.387796 .789041-5.387796 .73325-5.220423L.247073-3.841594C.231133-3.801743 .207223-3.737983 .207223-3.690162C.207223-3.626401 .263014-3.57858 .326775-3.57858C.414446-3.57858 .430386-3.618431 .478207-3.753923C.932503-5.029141 1.163636-5.124782 2.375093-5.124782H2.685928C2.925031-5.124782 2.933001-5.116812 2.933001-5.053051C2.933001-5.029141 2.901121-4.869738 2.893151-4.837858L1.841096-.653549C1.769365-.350685 1.745455-.263014 .916563-.263014C.661519-.263014 .581818-.263014 .581818-.111582C.581818-.103611 .581818 0 .71731 0C.932503 0 1.482441-.02391 1.697634-.02391H2.375093C2.598257-.02391 3.156164 0 3.379328 0C3.443088 0 3.56264 0 3.56264-.151432C3.56264-.263014 3.474969-.263014 3.259776-.263014C3.068493-.263014 3.004732-.263014 2.797509-.278954C2.542466-.302864 2.510585-.334745 2.510585-.438356C2.510585-.470237 2.518555-.502117 2.542466-.581818L3.602491-4.821918Z'/> +<path id='g3-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g3-106' d='M3.291656-4.97335C3.291656-5.124782 3.172105-5.276214 2.980822-5.276214C2.741719-5.276214 2.534496-5.053051 2.534496-4.845828C2.534496-4.694396 2.654047-4.542964 2.84533-4.542964C3.084433-4.542964 3.291656-4.766127 3.291656-4.97335ZM1.625903 .398506C1.506351 .884682 1.115816 1.40274 .629639 1.40274C.502117 1.40274 .382565 1.370859 .366625 1.362889C.613699 1.243337 .645579 1.028144 .645579 .956413C.645579 .765131 .502117 .661519 .334745 .661519C.103611 .661519-.111582 .860772-.111582 1.123786C-.111582 1.42665 .183313 1.625903 .637609 1.625903C1.123786 1.625903 2.000498 1.323039 2.239601 .366625L2.956912-2.486675C2.980822-2.582316 2.996762-2.646077 2.996762-2.765629C2.996762-3.203985 2.646077-3.514819 2.183811-3.514819C1.338979-3.514819 .844832-2.399004 .844832-2.295392C.844832-2.223661 .900623-2.191781 .964384-2.191781C1.052055-2.191781 1.060025-2.215691 1.115816-2.335243C1.354919-2.885181 1.761395-3.291656 2.1599-3.291656C2.327273-3.291656 2.422914-3.180075 2.422914-2.917061C2.422914-2.805479 2.399004-2.693898 2.375093-2.582316L1.625903 .398506Z'/> +<path id='g4-18' d='M4.963635-5.454544C4.963635-6.174544 4.767271-7.690907 3.654544-7.690907C2.138181-7.690907 .458182-4.614544 .458182-2.116363C.458182-1.090909 .774545 .12 1.767272 .12C3.305454 .12 4.963635-3.010908 4.963635-5.454544ZM1.614545-3.959999C1.799999-4.658181 2.018181-5.530908 2.454545-6.305453C2.74909-6.839998 3.152726-7.450907 3.643635-7.450907C4.178181-7.450907 4.243635-6.752725 4.243635-6.130907C4.243635-5.596362 4.156362-5.039999 3.894544-3.959999H1.614545ZM3.796363-3.610908C3.676363-3.10909 3.447272-2.181818 3.032726-1.396363C2.650908-.654545 2.236363-.12 1.767272-.12C1.407272-.12 1.178181-.436364 1.178181-1.450909C1.178181-1.90909 1.243636-2.541817 1.527272-3.610908H3.796363Z'/> +<path id='g4-88' d='M5.290908-4.483635L4.374544-6.654544C4.341817-6.741816 4.319999-6.785453 4.319999-6.796362C4.319999-6.861816 4.505453-7.069089 4.963635-7.112725C5.072726-7.123634 5.181817-7.134543 5.181817-7.319998C5.181817-7.450907 5.050908-7.450907 5.01818-7.450907C4.570908-7.450907 4.101817-7.41818 3.643635-7.41818C3.370908-7.41818 2.694545-7.450907 2.421818-7.450907C2.356363-7.450907 2.225454-7.450907 2.225454-7.232725C2.225454-7.112725 2.334545-7.112725 2.476363-7.112725C3.130908-7.112725 3.196363-7.003634 3.294545-6.763634L4.581817-3.719999L2.279999-1.243636L2.138181-1.123636C1.603636-.545454 1.090909-.370909 .534545-.338182C.392727-.327273 .294545-.327273 .294545-.12C.294545-.109091 .294545 0 .436364 0C.763636 0 1.123636-.032727 1.461818-.032727C1.865454-.032727 2.290908 0 2.683636 0C2.74909 0 2.879999 0 2.879999-.218182C2.879999-.327273 2.770908-.338182 2.74909-.338182C2.650908-.349091 2.312727-.370909 2.312727-.676363C2.312727-.850909 2.476363-1.025454 2.607272-1.167272L3.719999-2.345454L4.701817-3.414545L5.803635-.807273C5.847271-.687273 5.85818-.676363 5.85818-.654545C5.85818-.567273 5.650908-.381818 5.225453-.338182C5.105453-.327273 5.007271-.316364 5.007271-.130909C5.007271 0 5.127271 0 5.170908 0C5.476362 0 6.239998-.032727 6.545453-.032727C6.81818-.032727 7.483634 0 7.756361 0C7.832725 0 7.963634 0 7.963634-.207273C7.963634-.338182 7.854543-.338182 7.767271-.338182C7.036362-.349091 7.014543-.381818 6.829089-.818182C6.403635-1.832727 5.672726-3.534544 5.421817-4.18909C6.163635-4.952726 7.309089-6.250907 7.65818-6.556362C7.974543-6.81818 8.389089-7.079998 9.043634-7.112725C9.185452-7.123634 9.283634-7.123634 9.283634-7.330907C9.283634-7.341816 9.283634-7.450907 9.141816-7.450907C8.814543-7.450907 8.454543-7.41818 8.116361-7.41818C7.712725-7.41818 7.29818-7.450907 6.905453-7.450907C6.839998-7.450907 6.69818-7.450907 6.69818-7.232725C6.69818-7.156362 6.752725-7.123634 6.829089-7.112725C6.927271-7.101816 7.265453-7.079998 7.265453-6.774544C7.265453-6.621816 7.145453-6.479998 7.05818-6.381816L5.290908-4.483635Z'/> +<path id='g4-98' d='M2.607272-7.450907C2.607272-7.461816 2.607272-7.570907 2.465454-7.570907C2.214545-7.570907 1.418181-7.483634 1.134545-7.461816C1.047272-7.450907 .927272-7.439998 .927272-7.243634C.927272-7.112725 1.025454-7.112725 1.189091-7.112725C1.712727-7.112725 1.734545-7.036362 1.734545-6.927271C1.734545-6.850907 1.636363-6.479998 1.581818-6.250907L.687273-2.694545C.556363-2.14909 .512727-1.974545 .512727-1.592727C.512727-.556363 1.090909 .12 1.898181 .12C3.185454 .12 4.527271-1.505454 4.527271-3.076363C4.527271-4.06909 3.94909-4.821817 3.076363-4.821817C2.574545-4.821817 2.127272-4.505453 1.799999-4.167272L2.607272-7.450907ZM1.581818-3.327272C1.647272-3.567272 1.647272-3.58909 1.745454-3.70909C2.279999-4.418181 2.770908-4.581817 3.054545-4.581817C3.447272-4.581817 3.741817-4.254544 3.741817-3.556363C3.741817-2.912726 3.381817-1.658181 3.185454-1.243636C2.825454-.512727 2.323636-.12 1.898181-.12C1.527272-.12 1.167272-.414545 1.167272-1.221818C1.167272-1.429091 1.167272-1.636363 1.341818-2.323636L1.581818-3.327272Z'/> +<path id='g4-101' d='M2.039999-2.519999C2.356363-2.519999 3.163635-2.541817 3.70909-2.770908C4.472726-3.098181 4.527271-3.741817 4.527271-3.894544C4.527271-4.374544 4.112726-4.821817 3.359999-4.821817C2.14909-4.821817 .501818-3.763635 .501818-1.854545C.501818-.741818 1.145454 .12 2.214545 .12C3.774544 .12 4.690908-1.036363 4.690908-1.167272C4.690908-1.232727 4.625453-1.309091 4.559999-1.309091C4.505453-1.309091 4.483635-1.287272 4.418181-1.2C3.556363-.12 2.367272-.12 2.236363-.12C1.385454-.12 1.287272-1.036363 1.287272-1.385454C1.287272-1.516363 1.298181-1.854545 1.461818-2.519999H2.039999ZM1.527272-2.759999C1.952727-4.418181 3.076363-4.581817 3.359999-4.581817C3.872726-4.581817 4.167272-4.265453 4.167272-3.894544C4.167272-2.759999 2.421818-2.759999 1.974545-2.759999H1.527272Z'/> +<path id='g4-102' d='M4.003635-4.363635H4.941817C5.159999-4.363635 5.269089-4.363635 5.269089-4.581817C5.269089-4.701817 5.159999-4.701817 4.974544-4.701817H4.06909L4.298181-5.945453C4.341817-6.174544 4.494544-6.949089 4.559999-7.079998C4.658181-7.287271 4.843635-7.450907 5.072726-7.450907C5.116362-7.450907 5.399998-7.450907 5.607271-7.254543C5.127271-7.210907 5.01818-6.829089 5.01818-6.665453C5.01818-6.414544 5.214544-6.283635 5.421817-6.283635C5.705453-6.283635 6.021817-6.523635 6.021817-6.93818C6.021817-7.439998 5.519998-7.690907 5.072726-7.690907C4.701817-7.690907 4.014544-7.494543 3.687272-6.414544C3.621817-6.185453 3.58909-6.076362 3.327272-4.701817H2.574545C2.367272-4.701817 2.247272-4.701817 2.247272-4.494544C2.247272-4.363635 2.345454-4.363635 2.552727-4.363635H3.272726L2.454545-.054545C2.258181 1.003636 2.072727 1.996363 1.505454 1.996363C1.461818 1.996363 1.189091 1.996363 .981818 1.799999C1.483636 1.767272 1.581818 1.374545 1.581818 1.210909C1.581818 .96 1.385454 .829091 1.178181 .829091C.894545 .829091 .578182 1.069091 .578182 1.483636C.578182 1.974545 1.058182 2.236363 1.505454 2.236363C2.105454 2.236363 2.541817 1.592727 2.738181 1.178181C3.087272 .490909 3.338181-.829091 3.34909-.905454L4.003635-4.363635Z'/> +<path id='g4-103' d='M5.13818-4.112726C5.149089-4.178181 5.170908-4.232726 5.170908-4.30909C5.170908-4.494544 5.039999-4.603635 4.854544-4.603635C4.745453-4.603635 4.450908-4.527271 4.407271-4.134544C4.210908-4.538181 3.82909-4.821817 3.392726-4.821817C2.14909-4.821817 .796363-3.294545 .796363-1.723636C.796363-.643636 1.461818 0 2.247272 0C2.890908 0 3.403635-.512727 3.512726-.632727L3.523635-.621818C3.294545 .349091 3.163635 .796363 3.163635 .818182C3.119999 .916363 2.74909 1.996363 1.592727 1.996363C1.385454 1.996363 1.025454 1.985454 .72 1.887272C1.047272 1.78909 1.167272 1.505454 1.167272 1.32C1.167272 1.145454 1.047272 .938182 .752727 .938182C.512727 .938182 .163636 1.134545 .163636 1.570909C.163636 2.018181 .567273 2.236363 1.614545 2.236363C2.978181 2.236363 3.763635 1.385454 3.927272 .730909L5.13818-4.112726ZM3.719999-1.396363C3.654544-1.112727 3.403635-.84 3.163635-.632727C2.934545-.436364 2.596363-.24 2.279999-.24C1.734545-.24 1.570909-.807273 1.570909-1.243636C1.570909-1.767272 1.887272-3.054545 2.181818-3.610908C2.476363-4.145453 2.945454-4.581817 3.403635-4.581817C4.123635-4.581817 4.276362-3.698181 4.276362-3.643635S4.254544-3.523635 4.243635-3.479999L3.719999-1.396363Z'/> +<path id='g4-105' d='M3.098181-6.81818C3.098181-7.036362 2.945454-7.210907 2.694545-7.210907C2.399999-7.210907 2.105454-6.927271 2.105454-6.632725C2.105454-6.425453 2.258181-6.239998 2.519999-6.239998C2.770908-6.239998 3.098181-6.490907 3.098181-6.81818ZM2.26909-2.716363C2.399999-3.032726 2.399999-3.054545 2.50909-3.34909C2.596363-3.567272 2.650908-3.719999 2.650908-3.927272C2.650908-4.418181 2.301818-4.821817 1.756363-4.821817C.730909-4.821817 .316364-3.239999 .316364-3.141817C.316364-3.032726 .425454-3.032726 .447273-3.032726C.556363-3.032726 .567273-3.054545 .621818-3.22909C.916363-4.254544 1.352727-4.581817 1.723636-4.581817C1.810909-4.581817 1.996363-4.581817 1.996363-4.232726C1.996363-4.003635 1.919999-3.774544 1.876363-3.665454C1.78909-3.381817 1.298181-2.116363 1.123636-1.647272C1.014545-1.363636 .872727-1.003636 .872727-.774545C.872727-.261818 1.243636 .12 1.767272 .12C2.792726 .12 3.196363-1.461818 3.196363-1.56C3.196363-1.66909 3.098181-1.66909 3.065454-1.66909C2.956363-1.66909 2.956363-1.636363 2.901817-1.472727C2.705454-.785454 2.345454-.12 1.78909-.12C1.603636-.12 1.527272-.229091 1.527272-.48C1.527272-.752727 1.592727-.905454 1.843636-1.570909L2.26909-2.716363Z'/> +<path id='g4-108' d='M2.814545-7.450907C2.814545-7.461816 2.814545-7.570907 2.672727-7.570907C2.421818-7.570907 1.625454-7.483634 1.341818-7.461816C1.254545-7.450907 1.134545-7.439998 1.134545-7.232725C1.134545-7.112725 1.243636-7.112725 1.407272-7.112725C1.930909-7.112725 1.941818-7.014543 1.941818-6.927271L1.90909-6.709089L.534545-1.254545C.501818-1.134545 .48-1.058182 .48-.883636C.48-.261818 .96 .12 1.472727 .12C1.832727 .12 2.105454-.098182 2.290908-.490909C2.487272-.905454 2.618181-1.538181 2.618181-1.56C2.618181-1.66909 2.519999-1.66909 2.487272-1.66909C2.378181-1.66909 2.367272-1.625454 2.334545-1.472727C2.14909-.763636 1.941818-.12 1.505454-.12C1.178181-.12 1.178181-.469091 1.178181-.621818C1.178181-.883636 1.189091-.938182 1.243636-1.145454L2.814545-7.450907Z'/> +<path id='g4-109' d='M.96-.643636C.927272-.48 .861818-.229091 .861818-.174545C.861818 .021818 1.014545 .12 1.178181 .12C1.309091 .12 1.505454 .032727 1.581818-.185454C1.592727-.207273 1.723636-.72 1.78909-.992727L2.02909-1.974545C2.094545-2.214545 2.159999-2.454545 2.214545-2.705454C2.258181-2.890908 2.345454-3.207272 2.356363-3.250908C2.519999-3.58909 3.098181-4.581817 4.134544-4.581817C4.625453-4.581817 4.723635-4.178181 4.723635-3.818181C4.723635-3.545454 4.647271-3.239999 4.559999-2.912726L4.254544-1.647272L4.036363-.818182C3.992726-.6 3.894544-.229091 3.894544-.174545C3.894544 .021818 4.047272 .12 4.210908 .12C4.54909 .12 4.614544-.152727 4.701817-.501818C4.854544-1.112727 5.25818-2.705454 5.356362-3.130908C5.389089-3.272726 5.967271-4.581817 7.156362-4.581817C7.625452-4.581817 7.745452-4.210908 7.745452-3.818181C7.745452-3.196363 7.287271-1.952727 7.069089-1.374545C6.970907-1.112727 6.927271-.992727 6.927271-.774545C6.927271-.261818 7.309089 .12 7.821816 .12C8.84727 .12 9.250907-1.472727 9.250907-1.56C9.250907-1.66909 9.152725-1.66909 9.119997-1.66909C9.010907-1.66909 9.010907-1.636363 8.956361-1.472727C8.792725-.894545 8.443634-.12 7.843634-.12C7.65818-.12 7.581816-.229091 7.581816-.48C7.581816-.752727 7.679998-1.014545 7.77818-1.254545C7.985452-1.821818 8.443634-3.032726 8.443634-3.654544C8.443634-4.363635 8.00727-4.821817 7.189089-4.821817S5.814544-4.341817 5.410908-3.763635C5.399998-3.905453 5.367271-4.276362 5.061817-4.538181C4.78909-4.767271 4.439999-4.821817 4.167272-4.821817C3.185454-4.821817 2.650908-4.123635 2.465454-3.872726C2.410908-4.494544 1.952727-4.821817 1.461818-4.821817C.96-4.821817 .752727-4.396362 .654545-4.199999C.458182-3.818181 .316364-3.174545 .316364-3.141817C.316364-3.032726 .425454-3.032726 .447273-3.032726C.556363-3.032726 .567273-3.043636 .632727-3.283635C.818182-4.058181 1.036363-4.581817 1.429091-4.581817C1.603636-4.581817 1.767272-4.494544 1.767272-4.079999C1.767272-3.850908 1.734545-3.730908 1.592727-3.163635L.96-.643636Z'/> +<path id='g4-110' d='M.96-.643636C.927272-.48 .861818-.229091 .861818-.174545C.861818 .021818 1.014545 .12 1.178181 .12C1.309091 .12 1.505454 .032727 1.581818-.185454C1.592727-.207273 1.723636-.72 1.78909-.992727L2.02909-1.974545C2.094545-2.214545 2.159999-2.454545 2.214545-2.705454C2.258181-2.890908 2.345454-3.207272 2.356363-3.250908C2.519999-3.58909 3.098181-4.581817 4.134544-4.581817C4.625453-4.581817 4.723635-4.178181 4.723635-3.818181C4.723635-3.141817 4.18909-1.745454 4.014544-1.276363C3.916363-1.025454 3.905453-.894545 3.905453-.774545C3.905453-.261818 4.287272 .12 4.799999 .12C5.825453 .12 6.229089-1.472727 6.229089-1.56C6.229089-1.66909 6.130907-1.66909 6.09818-1.66909C5.989089-1.66909 5.989089-1.636363 5.934544-1.472727C5.716362-.730909 5.356362-.12 4.821817-.12C4.636362-.12 4.559999-.229091 4.559999-.48C4.559999-.752727 4.658181-1.014545 4.756362-1.254545C4.963635-1.832727 5.421817-3.032726 5.421817-3.654544C5.421817-4.385453 4.952726-4.821817 4.167272-4.821817C3.185454-4.821817 2.650908-4.123635 2.465454-3.872726C2.410908-4.483635 1.963636-4.821817 1.461818-4.821817S.752727-4.396362 .643636-4.199999C.469091-3.82909 .316364-3.185454 .316364-3.141817C.316364-3.032726 .425454-3.032726 .447273-3.032726C.556363-3.032726 .567273-3.043636 .632727-3.283635C.818182-4.058181 1.036363-4.581817 1.429091-4.581817C1.647272-4.581817 1.767272-4.439999 1.767272-4.079999C1.767272-3.850908 1.734545-3.730908 1.592727-3.163635L.96-.643636Z'/> +<path id='g4-111' d='M5.116362-2.978181C5.116362-4.112726 4.352726-4.821817 3.370908-4.821817C1.90909-4.821817 .447273-3.272726 .447273-1.723636C.447273-.643636 1.178181 .12 2.192727 .12C3.643635 .12 5.116362-1.385454 5.116362-2.978181ZM2.203636-.12C1.734545-.12 1.254545-.458182 1.254545-1.309091C1.254545-1.843636 1.538181-3.021817 1.887272-3.578181C2.432727-4.418181 3.054545-4.581817 3.359999-4.581817C3.992726-4.581817 4.319999-4.058181 4.319999-3.403635C4.319999-2.978181 4.101817-1.832727 3.687272-1.123636C3.305454-.490909 2.705454-.12 2.203636-.12Z'/> +<path id='g4-122' d='M1.450909-.905454C2.039999-1.538181 2.356363-1.810909 2.74909-2.14909C2.74909-2.159999 3.425454-2.738181 3.818181-3.130908C4.854544-4.145453 5.094544-4.66909 5.094544-4.712726C5.094544-4.821817 4.996362-4.821817 4.974544-4.821817C4.89818-4.821817 4.865453-4.799999 4.810908-4.701817C4.483635-4.178181 4.254544-4.003635 3.992726-4.003635S3.599999-4.167272 3.436363-4.352726C3.22909-4.603635 3.043636-4.821817 2.683636-4.821817C1.865454-4.821817 1.363636-3.807272 1.363636-3.578181C1.363636-3.523635 1.396363-3.458181 1.494545-3.458181S1.614545-3.512726 1.636363-3.578181C1.843636-4.079999 2.476363-4.090908 2.563636-4.090908C2.792726-4.090908 2.999999-4.014544 3.250908-3.927272C3.687272-3.763635 3.807272-3.763635 4.090908-3.763635C3.698181-3.294545 2.781817-2.50909 2.574545-2.334545L1.592727-1.418181C.850909-.687273 .469091-.065455 .469091 .010909C.469091 .12 .578182 .12 .6 .12C.687273 .12 .709091 .098182 .774545-.021818C1.025454-.403636 1.352727-.698182 1.701818-.698182C1.952727-.698182 2.061818-.6 2.334545-.283636C2.519999-.054545 2.716363 .12 3.032726 .12C4.112726 .12 4.745453-1.265454 4.745453-1.56C4.745453-1.614545 4.701817-1.66909 4.614544-1.66909C4.516362-1.66909 4.494544-1.603636 4.461817-1.527272C4.210908-.818182 3.512726-.610909 3.152726-.610909C2.934545-.610909 2.738181-.676363 2.50909-.752727C2.138181-.894545 1.974545-.938182 1.745454-.938182C1.723636-.938182 1.54909-.938182 1.450909-.905454Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -65.938653)'> +<use x='70.734745' y='68.742217' xlink:href='#g4-109'/> +<use x='80.313072' y='68.742217' xlink:href='#g4-105'/> +<use x='84.071394' y='68.742217' xlink:href='#g4-110'/> +<use x='90.61941' y='68.742217' xlink:href='#g4-105'/> +<use x='94.377732' y='68.742217' xlink:href='#g4-109'/> +<use x='103.95606' y='68.742217' xlink:href='#g4-105'/> +<use x='107.714382' y='68.742217' xlink:href='#g4-122'/> +<use x='113.267434' y='68.742217' xlink:href='#g4-101'/> +<use x='120.165145' y='60.560344' xlink:href='#g0-80'/> +<use x='131.680328' y='63.43491' xlink:href='#g5-49'/> +<use x='135.914511' y='63.43491' xlink:href='#g5-48'/> +<use x='140.148694' y='63.43491' xlink:href='#g3-59'/> +<use x='142.501017' y='63.43491' xlink:href='#g5-48'/> +<use x='146.7352' y='63.43491' xlink:href='#g5-48'/> +<use x='150.969383' y='63.43491' xlink:href='#g5-48'/> +<use x='131.680328' y='71.999001' xlink:href='#g3-105'/> +<use x='134.563468' y='71.999001' xlink:href='#g5-61'/> +<use x='141.149974' y='71.999001' xlink:href='#g5-49'/> +<use x='157.519846' y='60.560344' xlink:href='#g0-80'/> +<use x='169.035029' y='63.43491' xlink:href='#g5-49'/> +<use x='173.269212' y='63.43491' xlink:href='#g5-48'/> +<use x='177.503395' y='63.43491' xlink:href='#g3-59'/> +<use x='179.855719' y='63.43491' xlink:href='#g5-48'/> +<use x='184.089902' y='63.43491' xlink:href='#g5-48'/> +<use x='188.324085' y='63.43491' xlink:href='#g5-48'/> +<use x='169.035029' y='71.999001' xlink:href='#g3-106'/> +<use x='172.919054' y='71.999001' xlink:href='#g5-61'/> +<use x='179.505561' y='71.999001' xlink:href='#g5-49'/> +<use x='194.874548' y='68.742217' xlink:href='#g4-102'/> +<use x='201.38973' y='68.742217' xlink:href='#g6-40'/> +<use x='205.632168' y='68.742217' xlink:href='#g4-88'/> +<use x='214.670064' y='70.378567' xlink:href='#g3-105'/> +<use x='217.553203' y='70.378567' xlink:href='#g3-106'/> +<use x='221.93536' y='68.742217' xlink:href='#g6-41'/> +<use x='226.177799' y='68.742217' xlink:href='#g6-40'/> +<use x='230.420237' y='68.742217' xlink:href='#g4-18'/> +<use x='235.844478' y='64.783399' xlink:href='#g3-84'/> +<use x='235.541446' y='71.797745' xlink:href='#g3-105'/> +<use x='242.448968' y='68.742217' xlink:href='#g4-101'/> +<use x='247.528531' y='70.378567' xlink:href='#g3-106'/> +<use x='254.334886' y='68.742217' xlink:href='#g6-43'/> +<use x='265.24396' y='68.742217' xlink:href='#g4-98'/> +<use x='269.925778' y='70.378567' xlink:href='#g3-106'/> +<use x='276.732133' y='68.742217' xlink:href='#g6-43'/> +<use x='287.641207' y='68.742217' xlink:href='#g4-98'/> +<use x='292.323025' y='70.378567' xlink:href='#g3-106'/> +<use x='296.705178' y='64.32131' xlink:href='#g1-48'/> +<use x='301.924452' y='68.742217' xlink:href='#g2-0'/> +<use x='312.833526' y='68.742217' xlink:href='#g4-108'/> +<use x='316.303243' y='68.742217' xlink:href='#g4-111'/> +<use x='321.59113' y='68.742217' xlink:href='#g4-103'/> +<use x='327.185824' y='68.742217' xlink:href='#g4-88'/> +<use x='336.22372' y='70.378567' xlink:href='#g3-105'/> +<use x='339.10686' y='70.378567' xlink:href='#g3-106'/> +<use x='343.489017' y='68.742217' xlink:href='#g6-41'/> +<use x='347.731455' y='64.783399' xlink:href='#g5-50'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/gradient-descent.jpeg b/_resources/gradient-descent.jpeg new file mode 100644 index 0000000..64ca008 Binary files /dev/null and b/_resources/gradient-descent.jpeg differ diff --git a/_resources/human-performance.png b/_resources/human-performance.png new file mode 100644 index 0000000..fd65d71 Binary files /dev/null and b/_resources/human-performance.png differ diff --git a/_resources/image-caption.png b/_resources/image-caption.png new file mode 100644 index 0000000..bb4a3ae Binary files /dev/null and b/_resources/image-caption.png differ diff --git a/_resources/improve-performance.png b/_resources/improve-performance.png new file mode 100644 index 0000000..5461821 Binary files /dev/null and b/_resources/improve-performance.png differ diff --git a/_resources/inception-computation.png b/_resources/inception-computation.png new file mode 100644 index 0000000..9e53b99 Binary files /dev/null and b/_resources/inception-computation.png differ diff --git a/_resources/inception-module.png b/_resources/inception-module.png new file mode 100644 index 0000000..3e88c20 Binary files /dev/null and b/_resources/inception-module.png differ diff --git a/_resources/inception-motivation.png b/_resources/inception-motivation.png new file mode 100644 index 0000000..58bd530 Binary files /dev/null and b/_resources/inception-motivation.png differ diff --git a/_resources/inception-network.png b/_resources/inception-network.png new file mode 100644 index 0000000..e2c4cac Binary files /dev/null and b/_resources/inception-network.png differ diff --git a/_resources/inception.png b/_resources/inception.png new file mode 100644 index 0000000..c9d0a7d Binary files /dev/null and b/_resources/inception.png differ diff --git a/_resources/layer_activations.svg b/_resources/layer_activations.svg new file mode 100644 index 0000000..4a435e1 --- /dev/null +++ b/_resources/layer_activations.svg @@ -0,0 +1,36 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='111.982159pt' height='17.511011pt' viewBox='-.239051 -.22797 111.982159 17.511011'> +<defs> +<path id='g1-72' d='M6.36812-4.798007C6.447821-5.108842 6.463761-5.180573 7.021669-5.180573C7.204981-5.180573 7.300623-5.180573 7.300623-5.332005C7.300623-5.339975 7.292653-5.443587 7.165131-5.443587C7.013699-5.443587 6.822416-5.427646 6.670984-5.419676H6.168867C5.403736-5.419676 5.196513-5.443587 5.140722-5.443587C5.108842-5.443587 4.98132-5.443587 4.98132-5.292154C4.98132-5.180573 5.084932-5.180573 5.212453-5.180573C5.475467-5.180573 5.7066-5.180573 5.7066-5.053051C5.7066-5.021171 5.69863-5.0132 5.67472-4.909589L5.196513-2.964882H2.630137L3.092403-4.821918C3.164134-5.108842 3.180075-5.180573 3.761893-5.180573C3.921295-5.180573 4.016936-5.180573 4.016936-5.332005C4.016936-5.339975 4.008966-5.443587 3.881445-5.443587C3.730012-5.443587 3.53873-5.427646 3.387298-5.419676H2.885181C2.12005-5.419676 1.912827-5.443587 1.857036-5.443587C1.825156-5.443587 1.697634-5.443587 1.697634-5.292154C1.697634-5.180573 1.801245-5.180573 1.928767-5.180573C2.191781-5.180573 2.422914-5.180573 2.422914-5.053051C2.422914-5.021171 2.414944-5.0132 2.391034-4.909589L1.323039-.629639C1.251308-.326775 1.235367-.263014 .645579-.263014C.494147-.263014 .398506-.263014 .398506-.111582C.398506-.079701 .422416 0 .526027 0C.67746 0 .868742-.01594 1.020174-.02391H1.530262C2.295392-.02391 2.502615 0 2.558406 0C2.606227 0 2.717808 0 2.717808-.151432C2.717808-.263014 2.614197-.263014 2.478705-.263014C2.454795-.263014 2.311333-.263014 2.175841-.278954C2.008468-.294894 1.992528-.318804 1.992528-.390535C1.992528-.430386 2.008468-.478207 2.016438-.518057L2.558406-2.701868H5.124782C4.893649-1.761395 4.582814-.518057 4.574844-.518057C4.503113-.270984 4.359651-.270984 3.881445-.263014C3.785803-.263014 3.682192-.263014 3.682192-.111582C3.682192-.079701 3.706102 0 3.809714 0C3.961146 0 4.152428-.01594 4.303861-.02391H4.813948C5.579078-.02391 5.786301 0 5.842092 0C5.889913 0 6.001494 0 6.001494-.151432C6.001494-.263014 5.897883-.263014 5.762391-.263014C5.738481-.263014 5.595019-.263014 5.459527-.278954C5.292154-.294894 5.276214-.318804 5.276214-.390535C5.276214-.430386 5.292154-.478207 5.300125-.518057L6.36812-4.798007Z'/> +<path id='g1-87' d='M7.7868-4.487173C8.145455-5.084932 8.416438-5.156663 8.679452-5.180573C8.751183-5.188543 8.846824-5.196513 8.846824-5.332005C8.846824-5.387796 8.799004-5.443587 8.735243-5.443587C8.599751-5.443587 8.727273-5.419676 8.097634-5.419676C7.571606-5.419676 7.340473-5.443587 7.300623-5.443587C7.173101-5.443587 7.149191-5.355915 7.149191-5.292154C7.149191-5.188543 7.260772-5.180573 7.284682-5.180573C7.507846-5.172603 7.707098-5.092902 7.707098-4.893649C7.707098-4.790037 7.627397-4.654545 7.587547-4.590785L5.379826-.876712L5.00523-4.837858C4.99726-4.853798 4.98929-4.909589 4.98929-4.933499C4.98929-5.053051 5.140722-5.180573 5.531258-5.180573C5.650809-5.180573 5.746451-5.180573 5.746451-5.332005C5.746451-5.395766 5.69863-5.443587 5.618929-5.443587S5.244334-5.427646 5.164633-5.419676H4.686426C3.977086-5.419676 3.889415-5.443587 3.825654-5.443587C3.793773-5.443587 3.666252-5.443587 3.666252-5.292154C3.666252-5.180573 3.769863-5.180573 3.889415-5.180573C4.224159-5.180573 4.24807-5.116812 4.26401-5.00523C4.26401-4.98929 4.311831-4.527024 4.311831-4.503113S4.311831-4.439352 4.25604-4.351681L2.183811-.876712L1.809215-4.798007C1.809215-4.837858 1.801245-4.901619 1.801245-4.941469C1.801245-5.00523 1.865006-5.180573 2.351183-5.180573C2.438854-5.180573 2.550436-5.180573 2.550436-5.332005C2.550436-5.395766 2.494645-5.443587 2.422914-5.443587C2.343213-5.443587 2.048319-5.427646 1.968618-5.419676H1.490411C.757161-5.419676 .72528-5.443587 .629639-5.443587C.502117-5.443587 .478207-5.355915 .478207-5.292154C.478207-5.180573 .589788-5.180573 .67746-5.180573C1.052055-5.180573 1.060025-5.124782 1.075965-4.925529L1.538232-.063761C1.554172 .087671 1.562142 .167372 1.697634 .167372C1.761395 .167372 1.841096 .143462 1.928767 0L4.351681-4.080697L4.734247-.055791C4.750187 .087671 4.758157 .167372 4.885679 .167372C5.021171 .167372 5.068991 .079701 5.116812 0L7.7868-4.487173Z'/> +<path id='g1-99' d='M3.259776-3.052553C2.933001-3.012702 2.82939-2.765629 2.82939-2.606227C2.82939-2.375093 3.036613-2.311333 3.140224-2.311333C3.180075-2.311333 3.58655-2.343213 3.58655-2.82939S3.060523-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.541968 .868742 .079701 1.753425 .079701C3.012702 .079701 3.674222-.72528 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.769365-.143462C1.338979-.143462 .996264-.406476 .996264-1.012204C.996264-1.362889 1.155666-2.207721 1.530262-2.701868C1.880946-3.148194 2.279452-3.291656 2.590286-3.291656C2.685928-3.291656 3.052553-3.283686 3.259776-3.052553Z'/> +<path id='g1-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g3-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g3-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g0-2' d='M4.65056-3.323537L2.259527-5.702615C2.116065-5.846077 2.092154-5.869988 1.996513-5.869988C1.876961-5.869988 1.75741-5.762391 1.75741-5.630884C1.75741-5.547198 1.78132-5.523288 1.912827-5.391781L4.303861-2.988792L1.912827-.585803C1.78132-.454296 1.75741-.430386 1.75741-.3467C1.75741-.215193 1.876961-.107597 1.996513-.107597C2.092154-.107597 2.116065-.131507 2.259527-.274969L4.638605-2.654047L7.113325-.179328C7.137235-.167372 7.220922-.107597 7.292653-.107597C7.436115-.107597 7.531756-.215193 7.531756-.3467C7.531756-.37061 7.531756-.418431 7.49589-.478207C7.483935-.502117 5.583064-2.379078 4.985305-2.988792L7.173101-5.176588C7.232877-5.248319 7.412204-5.403736 7.47198-5.475467C7.483935-5.499377 7.531756-5.547198 7.531756-5.630884C7.531756-5.762391 7.436115-5.869988 7.292653-5.869988C7.197011-5.869988 7.149191-5.822167 7.017684-5.69066L4.65056-3.323537Z'/> +<path id='g2-109' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-4.124533 5.057036-3.88543 4.913574-3.323537L4.614695-2.092154C4.519054-1.733499 4.291905-.848817 4.267995-.71731C4.220174-.537983 4.148443-.227148 4.148443-.179328C4.148443-.011955 4.27995 .119552 4.459278 .119552C4.817933 .119552 4.877709-.155417 4.985305-.585803L5.702615-3.443088C5.726526-3.53873 6.348194-5.033126 7.663263-5.033126C8.141469-5.033126 8.308842-4.674471 8.308842-4.196264C8.308842-3.526775 7.84259-2.223661 7.579577-1.506351C7.47198-1.219427 7.412204-1.06401 7.412204-.848817C7.412204-.310834 7.782814 .119552 8.356663 .119552C9.468493 .119552 9.886924-1.637858 9.886924-1.709589C9.886924-1.769365 9.839103-1.817186 9.767372-1.817186C9.659776-1.817186 9.647821-1.78132 9.588045-1.578082C9.313076-.621669 8.870735-.119552 8.392528-.119552C8.272976-.119552 8.081694-.131507 8.081694-.514072C8.081694-.824907 8.225156-1.207472 8.272976-1.338979C8.488169-1.912827 9.026152-3.323537 9.026152-4.016936C9.026152-4.734247 8.607721-5.272229 7.699128-5.272229C6.898132-5.272229 6.252553-4.817933 5.774346-4.112578C5.738481-4.758157 5.34396-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.207472-5.272229 .932503-5.104857 .705355-4.65056C.490162-4.220174 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.812951-4.327771 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +<path id='g2-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -61.02)'> +<use x='56.413267' y='65.886243' xlink:href='#g2-109'/> +<use x='69.309198' y='65.886243' xlink:href='#g0-2'/> +<use x='81.264358' y='65.886243' xlink:href='#g2-110'/> +<use x='88.251964' y='59.775841' xlink:href='#g3-91'/> +<use x='90.604288' y='59.775841' xlink:href='#g1-108'/> +<use x='93.226413' y='59.775841' xlink:href='#g3-93'/> +<use x='88.251964' y='69.127354' xlink:href='#g1-72'/> +<use x='98.977064' y='65.886243' xlink:href='#g0-2'/> +<use x='110.932225' y='65.886243' xlink:href='#g2-110'/> +<use x='117.919831' y='59.775841' xlink:href='#g3-91'/> +<use x='120.272154' y='59.775841' xlink:href='#g1-108'/> +<use x='122.89428' y='59.775841' xlink:href='#g3-93'/> +<use x='117.919831' y='69.127354' xlink:href='#g1-87'/> +<use x='130.240389' y='65.886243' xlink:href='#g0-2'/> +<use x='142.195549' y='65.886243' xlink:href='#g2-110'/> +<use x='149.183155' y='60.950057' xlink:href='#g3-91'/> +<use x='151.535479' y='60.950057' xlink:href='#g1-108'/> +<use x='154.157604' y='60.950057' xlink:href='#g3-93'/> +<use x='149.183155' y='68.841758' xlink:href='#g1-99'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_bias.svg b/_resources/layer_bias.svg new file mode 100644 index 0000000..a742e15 --- /dev/null +++ b/_resources/layer_bias.svg @@ -0,0 +1,30 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='63.311234pt' height='15.762355pt' viewBox='-.239051 -.232383 63.311234 15.762355'> +<defs> +<path id='g0-99' d='M3.259776-3.052553C2.933001-3.012702 2.82939-2.765629 2.82939-2.606227C2.82939-2.375093 3.036613-2.311333 3.140224-2.311333C3.180075-2.311333 3.58655-2.343213 3.58655-2.82939S3.060523-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.541968 .868742 .079701 1.753425 .079701C3.012702 .079701 3.674222-.72528 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.769365-.143462C1.338979-.143462 .996264-.406476 .996264-1.012204C.996264-1.362889 1.155666-2.207721 1.530262-2.701868C1.880946-3.148194 2.279452-3.291656 2.590286-3.291656C2.685928-3.291656 3.052553-3.283686 3.259776-3.052553Z'/> +<path id='g0-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g2-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g2-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g1-59' d='M2.331258 .047821C2.331258-.645579 2.10411-1.159651 1.613948-1.159651C1.231382-1.159651 1.0401-.848817 1.0401-.585803S1.219427 0 1.625903 0C1.78132 0 1.912827-.047821 2.020423-.155417C2.044334-.179328 2.056289-.179328 2.068244-.179328C2.092154-.179328 2.092154-.011955 2.092154 .047821C2.092154 .442341 2.020423 1.219427 1.327024 1.996513C1.195517 2.139975 1.195517 2.163885 1.195517 2.187796C1.195517 2.247572 1.255293 2.307347 1.315068 2.307347C1.41071 2.307347 2.331258 1.422665 2.331258 .047821Z'/> +<path id='g1-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +<path id='g3-40' d='M3.88543 2.905106C3.88543 2.86924 3.88543 2.84533 3.682192 2.642092C2.486675 1.43462 1.817186-.537983 1.817186-2.976837C1.817186-5.296139 2.379078-7.292653 3.765878-8.703362C3.88543-8.810959 3.88543-8.834869 3.88543-8.870735C3.88543-8.942466 3.825654-8.966376 3.777833-8.966376C3.622416-8.966376 2.642092-8.105604 2.056289-6.933998C1.446575-5.726526 1.171606-4.447323 1.171606-2.976837C1.171606-1.912827 1.338979-.490162 1.960648 .789041C2.666002 2.223661 3.646326 3.000747 3.777833 3.000747C3.825654 3.000747 3.88543 2.976837 3.88543 2.905106Z'/> +<path id='g3-41' d='M3.371357-2.976837C3.371357-3.88543 3.251806-5.36787 2.582316-6.75467C1.876961-8.18929 .896638-8.966376 .765131-8.966376C.71731-8.966376 .657534-8.942466 .657534-8.870735C.657534-8.834869 .657534-8.810959 .860772-8.607721C2.056289-7.400249 2.725778-5.427646 2.725778-2.988792C2.725778-.669489 2.163885 1.327024 .777086 2.737733C.657534 2.84533 .657534 2.86924 .657534 2.905106C.657534 2.976837 .71731 3.000747 .765131 3.000747C.920548 3.000747 1.900872 2.139975 2.486675 .968369C3.096389-.251059 3.371357-1.542217 3.371357-2.976837Z'/> +<path id='g3-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.201193)'> +<use x='56.413267' y='65.753425' xlink:href='#g3-40'/> +<use x='60.965593' y='65.753425' xlink:href='#g3-49'/> +<use x='66.818583' y='65.753425' xlink:href='#g1-59'/> +<use x='72.062742' y='65.753425' xlink:href='#g3-49'/> +<use x='77.915732' y='65.753425' xlink:href='#g1-59'/> +<use x='83.159891' y='65.753425' xlink:href='#g3-49'/> +<use x='89.012881' y='65.753425' xlink:href='#g1-59'/> +<use x='94.25704' y='65.753425' xlink:href='#g1-110'/> +<use x='101.244646' y='60.817239' xlink:href='#g2-91'/> +<use x='103.59697' y='60.817239' xlink:href='#g0-108'/> +<use x='106.219095' y='60.817239' xlink:href='#g2-93'/> +<use x='101.244646' y='68.70894' xlink:href='#g0-99'/> +<use x='109.069551' y='65.753425' xlink:href='#g3-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_filter_shape.svg b/_resources/layer_filter_shape.svg new file mode 100644 index 0000000..31262d5 --- /dev/null +++ b/_resources/layer_filter_shape.svg @@ -0,0 +1,34 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='93.907513pt' height='15.762355pt' viewBox='-.239051 -.232383 93.907513 15.762355'> +<defs> +<path id='g0-0' d='M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z'/> +<path id='g1-2' d='M4.65056-3.323537L2.259527-5.702615C2.116065-5.846077 2.092154-5.869988 1.996513-5.869988C1.876961-5.869988 1.75741-5.762391 1.75741-5.630884C1.75741-5.547198 1.78132-5.523288 1.912827-5.391781L4.303861-2.988792L1.912827-.585803C1.78132-.454296 1.75741-.430386 1.75741-.3467C1.75741-.215193 1.876961-.107597 1.996513-.107597C2.092154-.107597 2.116065-.131507 2.259527-.274969L4.638605-2.654047L7.113325-.179328C7.137235-.167372 7.220922-.107597 7.292653-.107597C7.436115-.107597 7.531756-.215193 7.531756-.3467C7.531756-.37061 7.531756-.418431 7.49589-.478207C7.483935-.502117 5.583064-2.379078 4.985305-2.988792L7.173101-5.176588C7.232877-5.248319 7.412204-5.403736 7.47198-5.475467C7.483935-5.499377 7.531756-5.547198 7.531756-5.630884C7.531756-5.762391 7.436115-5.869988 7.292653-5.869988C7.197011-5.869988 7.149191-5.822167 7.017684-5.69066L4.65056-3.323537Z'/> +<path id='g2-99' d='M3.259776-3.052553C2.933001-3.012702 2.82939-2.765629 2.82939-2.606227C2.82939-2.375093 3.036613-2.311333 3.140224-2.311333C3.180075-2.311333 3.58655-2.343213 3.58655-2.82939S3.060523-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.541968 .868742 .079701 1.753425 .079701C3.012702 .079701 3.674222-.72528 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.769365-.143462C1.338979-.143462 .996264-.406476 .996264-1.012204C.996264-1.362889 1.155666-2.207721 1.530262-2.701868C1.880946-3.148194 2.279452-3.291656 2.590286-3.291656C2.685928-3.291656 3.052553-3.283686 3.259776-3.052553Z'/> +<path id='g2-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g4-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g4-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g4-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g3-102' d='M5.332005-4.805978C5.571108-4.805978 5.66675-4.805978 5.66675-5.033126C5.66675-5.152677 5.571108-5.152677 5.355915-5.152677H4.387547C4.614695-6.38406 4.782067-7.232877 4.877709-7.615442C4.94944-7.902366 5.200498-8.177335 5.511333-8.177335C5.762391-8.177335 6.01345-8.069738 6.133001-7.962142C5.66675-7.914321 5.523288-7.567621 5.523288-7.364384C5.523288-7.12528 5.702615-6.981818 5.929763-6.981818C6.168867-6.981818 6.527522-7.185056 6.527522-7.639352C6.527522-8.141469 6.025405-8.416438 5.499377-8.416438C4.985305-8.416438 4.483188-8.033873 4.244085-7.567621C4.028892-7.149191 3.90934-6.718804 3.634371-5.152677H2.833375C2.606227-5.152677 2.486675-5.152677 2.486675-4.937484C2.486675-4.805978 2.558406-4.805978 2.797509-4.805978H3.56264C3.347447-3.694147 2.857285-.992279 2.582316 .286924C2.379078 1.327024 2.199751 2.199751 1.601993 2.199751C1.566127 2.199751 1.219427 2.199751 1.004234 1.972603C1.613948 1.924782 1.613948 1.398755 1.613948 1.3868C1.613948 1.147696 1.43462 1.004234 1.207472 1.004234C.968369 1.004234 .609714 1.207472 .609714 1.661768C.609714 2.175841 1.135741 2.438854 1.601993 2.438854C2.82142 2.438854 3.323537 .251059 3.455044-.3467C3.670237-1.267248 4.25604-4.447323 4.315816-4.805978H5.332005Z'/> +<path id='g3-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.201193)'> +<use x='56.413267' y='65.753425' xlink:href='#g3-102'/> +<use x='63.459704' y='60.817239' xlink:href='#g4-91'/> +<use x='65.812028' y='60.817239' xlink:href='#g2-108'/> +<use x='68.434153' y='60.817239' xlink:href='#g4-93'/> +<use x='73.941272' y='65.753425' xlink:href='#g1-2'/> +<use x='85.896432' y='65.753425' xlink:href='#g3-102'/> +<use x='92.942869' y='60.817239' xlink:href='#g4-91'/> +<use x='95.295193' y='60.817239' xlink:href='#g2-108'/> +<use x='97.917318' y='60.817239' xlink:href='#g4-93'/> +<use x='103.424437' y='65.753425' xlink:href='#g1-2'/> +<use x='115.379598' y='65.753425' xlink:href='#g3-110'/> +<use x='122.367203' y='60.817239' xlink:href='#g4-91'/> +<use x='124.719527' y='60.817239' xlink:href='#g2-108'/> +<use x='127.341652' y='60.817239' xlink:href='#g0-0'/> +<use x='133.928159' y='60.817239' xlink:href='#g4-49'/> +<use x='138.162342' y='60.817239' xlink:href='#g4-93'/> +<use x='122.367203' y='68.70894' xlink:href='#g2-99'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_filter_size.svg b/_resources/layer_filter_size.svg new file mode 100644 index 0000000..e983408 --- /dev/null +++ b/_resources/layer_filter_size.svg @@ -0,0 +1,16 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='15.114659pt' height='15.088465pt' viewBox='-.239051 -.232383 15.114659 15.088465'> +<defs> +<path id='g0-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g2-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g2-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g1-102' d='M5.332005-4.805978C5.571108-4.805978 5.66675-4.805978 5.66675-5.033126C5.66675-5.152677 5.571108-5.152677 5.355915-5.152677H4.387547C4.614695-6.38406 4.782067-7.232877 4.877709-7.615442C4.94944-7.902366 5.200498-8.177335 5.511333-8.177335C5.762391-8.177335 6.01345-8.069738 6.133001-7.962142C5.66675-7.914321 5.523288-7.567621 5.523288-7.364384C5.523288-7.12528 5.702615-6.981818 5.929763-6.981818C6.168867-6.981818 6.527522-7.185056 6.527522-7.639352C6.527522-8.141469 6.025405-8.416438 5.499377-8.416438C4.985305-8.416438 4.483188-8.033873 4.244085-7.567621C4.028892-7.149191 3.90934-6.718804 3.634371-5.152677H2.833375C2.606227-5.152677 2.486675-5.152677 2.486675-4.937484C2.486675-4.805978 2.558406-4.805978 2.797509-4.805978H3.56264C3.347447-3.694147 2.857285-.992279 2.582316 .286924C2.379078 1.327024 2.199751 2.199751 1.601993 2.199751C1.566127 2.199751 1.219427 2.199751 1.004234 1.972603C1.613948 1.924782 1.613948 1.398755 1.613948 1.3868C1.613948 1.147696 1.43462 1.004234 1.207472 1.004234C.968369 1.004234 .609714 1.207472 .609714 1.661768C.609714 2.175841 1.135741 2.438854 1.601993 2.438854C2.82142 2.438854 3.323537 .251059 3.455044-.3467C3.670237-1.267248 4.25604-4.447323 4.315816-4.805978H5.332005Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.201193)'> +<use x='56.413267' y='65.753425' xlink:href='#g1-102'/> +<use x='63.459704' y='60.817239' xlink:href='#g2-91'/> +<use x='65.812028' y='60.817239' xlink:href='#g0-108'/> +<use x='68.434153' y='60.817239' xlink:href='#g2-93'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_input_shape.svg b/_resources/layer_input_shape.svg new file mode 100644 index 0000000..cdbd16d --- /dev/null +++ b/_resources/layer_input_shape.svg @@ -0,0 +1,41 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='118.229313pt' height='17.511011pt' viewBox='-.239051 -.22797 118.229313 17.511011'> +<defs> +<path id='g1-2' d='M4.65056-3.323537L2.259527-5.702615C2.116065-5.846077 2.092154-5.869988 1.996513-5.869988C1.876961-5.869988 1.75741-5.762391 1.75741-5.630884C1.75741-5.547198 1.78132-5.523288 1.912827-5.391781L4.303861-2.988792L1.912827-.585803C1.78132-.454296 1.75741-.430386 1.75741-.3467C1.75741-.215193 1.876961-.107597 1.996513-.107597C2.092154-.107597 2.116065-.131507 2.259527-.274969L4.638605-2.654047L7.113325-.179328C7.137235-.167372 7.220922-.107597 7.292653-.107597C7.436115-.107597 7.531756-.215193 7.531756-.3467C7.531756-.37061 7.531756-.418431 7.49589-.478207C7.483935-.502117 5.583064-2.379078 4.985305-2.988792L7.173101-5.176588C7.232877-5.248319 7.412204-5.403736 7.47198-5.475467C7.483935-5.499377 7.531756-5.547198 7.531756-5.630884C7.531756-5.762391 7.436115-5.869988 7.292653-5.869988C7.197011-5.869988 7.149191-5.822167 7.017684-5.69066L4.65056-3.323537Z'/> +<path id='g0-0' d='M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z'/> +<path id='g2-72' d='M6.36812-4.798007C6.447821-5.108842 6.463761-5.180573 7.021669-5.180573C7.204981-5.180573 7.300623-5.180573 7.300623-5.332005C7.300623-5.339975 7.292653-5.443587 7.165131-5.443587C7.013699-5.443587 6.822416-5.427646 6.670984-5.419676H6.168867C5.403736-5.419676 5.196513-5.443587 5.140722-5.443587C5.108842-5.443587 4.98132-5.443587 4.98132-5.292154C4.98132-5.180573 5.084932-5.180573 5.212453-5.180573C5.475467-5.180573 5.7066-5.180573 5.7066-5.053051C5.7066-5.021171 5.69863-5.0132 5.67472-4.909589L5.196513-2.964882H2.630137L3.092403-4.821918C3.164134-5.108842 3.180075-5.180573 3.761893-5.180573C3.921295-5.180573 4.016936-5.180573 4.016936-5.332005C4.016936-5.339975 4.008966-5.443587 3.881445-5.443587C3.730012-5.443587 3.53873-5.427646 3.387298-5.419676H2.885181C2.12005-5.419676 1.912827-5.443587 1.857036-5.443587C1.825156-5.443587 1.697634-5.443587 1.697634-5.292154C1.697634-5.180573 1.801245-5.180573 1.928767-5.180573C2.191781-5.180573 2.422914-5.180573 2.422914-5.053051C2.422914-5.021171 2.414944-5.0132 2.391034-4.909589L1.323039-.629639C1.251308-.326775 1.235367-.263014 .645579-.263014C.494147-.263014 .398506-.263014 .398506-.111582C.398506-.079701 .422416 0 .526027 0C.67746 0 .868742-.01594 1.020174-.02391H1.530262C2.295392-.02391 2.502615 0 2.558406 0C2.606227 0 2.717808 0 2.717808-.151432C2.717808-.263014 2.614197-.263014 2.478705-.263014C2.454795-.263014 2.311333-.263014 2.175841-.278954C2.008468-.294894 1.992528-.318804 1.992528-.390535C1.992528-.430386 2.008468-.478207 2.016438-.518057L2.558406-2.701868H5.124782C4.893649-1.761395 4.582814-.518057 4.574844-.518057C4.503113-.270984 4.359651-.270984 3.881445-.263014C3.785803-.263014 3.682192-.263014 3.682192-.111582C3.682192-.079701 3.706102 0 3.809714 0C3.961146 0 4.152428-.01594 4.303861-.02391H4.813948C5.579078-.02391 5.786301 0 5.842092 0C5.889913 0 6.001494 0 6.001494-.151432C6.001494-.263014 5.897883-.263014 5.762391-.263014C5.738481-.263014 5.595019-.263014 5.459527-.278954C5.292154-.294894 5.276214-.318804 5.276214-.390535C5.276214-.430386 5.292154-.478207 5.300125-.518057L6.36812-4.798007Z'/> +<path id='g2-87' d='M7.7868-4.487173C8.145455-5.084932 8.416438-5.156663 8.679452-5.180573C8.751183-5.188543 8.846824-5.196513 8.846824-5.332005C8.846824-5.387796 8.799004-5.443587 8.735243-5.443587C8.599751-5.443587 8.727273-5.419676 8.097634-5.419676C7.571606-5.419676 7.340473-5.443587 7.300623-5.443587C7.173101-5.443587 7.149191-5.355915 7.149191-5.292154C7.149191-5.188543 7.260772-5.180573 7.284682-5.180573C7.507846-5.172603 7.707098-5.092902 7.707098-4.893649C7.707098-4.790037 7.627397-4.654545 7.587547-4.590785L5.379826-.876712L5.00523-4.837858C4.99726-4.853798 4.98929-4.909589 4.98929-4.933499C4.98929-5.053051 5.140722-5.180573 5.531258-5.180573C5.650809-5.180573 5.746451-5.180573 5.746451-5.332005C5.746451-5.395766 5.69863-5.443587 5.618929-5.443587S5.244334-5.427646 5.164633-5.419676H4.686426C3.977086-5.419676 3.889415-5.443587 3.825654-5.443587C3.793773-5.443587 3.666252-5.443587 3.666252-5.292154C3.666252-5.180573 3.769863-5.180573 3.889415-5.180573C4.224159-5.180573 4.24807-5.116812 4.26401-5.00523C4.26401-4.98929 4.311831-4.527024 4.311831-4.503113S4.311831-4.439352 4.25604-4.351681L2.183811-.876712L1.809215-4.798007C1.809215-4.837858 1.801245-4.901619 1.801245-4.941469C1.801245-5.00523 1.865006-5.180573 2.351183-5.180573C2.438854-5.180573 2.550436-5.180573 2.550436-5.332005C2.550436-5.395766 2.494645-5.443587 2.422914-5.443587C2.343213-5.443587 2.048319-5.427646 1.968618-5.419676H1.490411C.757161-5.419676 .72528-5.443587 .629639-5.443587C.502117-5.443587 .478207-5.355915 .478207-5.292154C.478207-5.180573 .589788-5.180573 .67746-5.180573C1.052055-5.180573 1.060025-5.124782 1.075965-4.925529L1.538232-.063761C1.554172 .087671 1.562142 .167372 1.697634 .167372C1.761395 .167372 1.841096 .143462 1.928767 0L4.351681-4.080697L4.734247-.055791C4.750187 .087671 4.758157 .167372 4.885679 .167372C5.021171 .167372 5.068991 .079701 5.116812 0L7.7868-4.487173Z'/> +<path id='g2-99' d='M3.259776-3.052553C2.933001-3.012702 2.82939-2.765629 2.82939-2.606227C2.82939-2.375093 3.036613-2.311333 3.140224-2.311333C3.180075-2.311333 3.58655-2.343213 3.58655-2.82939S3.060523-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.541968 .868742 .079701 1.753425 .079701C3.012702 .079701 3.674222-.72528 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.769365-.143462C1.338979-.143462 .996264-.406476 .996264-1.012204C.996264-1.362889 1.155666-2.207721 1.530262-2.701868C1.880946-3.148194 2.279452-3.291656 2.590286-3.291656C2.685928-3.291656 3.052553-3.283686 3.259776-3.052553Z'/> +<path id='g2-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g4-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g4-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g4-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g3-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -61.02)'> +<use x='56.413267' y='65.886243' xlink:href='#g3-110'/> +<use x='63.400873' y='59.775841' xlink:href='#g4-91'/> +<use x='65.753197' y='59.775841' xlink:href='#g2-108'/> +<use x='68.375322' y='59.775841' xlink:href='#g0-0'/> +<use x='74.961828' y='59.775841' xlink:href='#g4-49'/> +<use x='79.196011' y='59.775841' xlink:href='#g4-93'/> +<use x='63.400873' y='69.127354' xlink:href='#g2-72'/> +<use x='84.70313' y='65.886243' xlink:href='#g1-2'/> +<use x='96.658291' y='65.886243' xlink:href='#g3-110'/> +<use x='103.645897' y='59.775841' xlink:href='#g4-91'/> +<use x='105.99822' y='59.775841' xlink:href='#g2-108'/> +<use x='108.620346' y='59.775841' xlink:href='#g0-0'/> +<use x='115.206852' y='59.775841' xlink:href='#g4-49'/> +<use x='119.441035' y='59.775841' xlink:href='#g4-93'/> +<use x='103.645897' y='69.127354' xlink:href='#g2-87'/> +<use x='124.948154' y='65.886243' xlink:href='#g1-2'/> +<use x='136.903315' y='65.886243' xlink:href='#g3-110'/> +<use x='143.890921' y='60.950057' xlink:href='#g4-91'/> +<use x='146.243244' y='60.950057' xlink:href='#g2-108'/> +<use x='148.865369' y='60.950057' xlink:href='#g0-0'/> +<use x='155.451876' y='60.950057' xlink:href='#g4-49'/> +<use x='159.686059' y='60.950057' xlink:href='#g4-93'/> +<use x='143.890921' y='68.841758' xlink:href='#g2-99'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_num_filters.svg b/_resources/layer_num_filters.svg new file mode 100644 index 0000000..8e6ebe1 --- /dev/null +++ b/_resources/layer_num_filters.svg @@ -0,0 +1,18 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='15.04818pt' height='15.762355pt' viewBox='-.239051 -.232383 15.04818 15.762355'> +<defs> +<path id='g0-99' d='M3.259776-3.052553C2.933001-3.012702 2.82939-2.765629 2.82939-2.606227C2.82939-2.375093 3.036613-2.311333 3.140224-2.311333C3.180075-2.311333 3.58655-2.343213 3.58655-2.82939S3.060523-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.541968 .868742 .079701 1.753425 .079701C3.012702 .079701 3.674222-.72528 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.769365-.143462C1.338979-.143462 .996264-.406476 .996264-1.012204C.996264-1.362889 1.155666-2.207721 1.530262-2.701868C1.880946-3.148194 2.279452-3.291656 2.590286-3.291656C2.685928-3.291656 3.052553-3.283686 3.259776-3.052553Z'/> +<path id='g0-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g2-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g2-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g1-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.201193)'> +<use x='56.413267' y='65.753425' xlink:href='#g1-110'/> +<use x='63.400873' y='60.817239' xlink:href='#g2-91'/> +<use x='65.753197' y='60.817239' xlink:href='#g0-108'/> +<use x='68.375322' y='60.817239' xlink:href='#g2-93'/> +<use x='63.400873' y='68.70894' xlink:href='#g0-99'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_output_height.svg b/_resources/layer_output_height.svg new file mode 100644 index 0000000..68fbdad --- /dev/null +++ b/_resources/layer_output_height.svg @@ -0,0 +1,64 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='136.281933pt' height='22.284864pt' viewBox='-.299738 -.244857 136.281933 22.284864'> +<defs> +<path id='g3-72' d='M5.302117-3.598506C5.349938-3.795766 5.361893-3.843587 5.8401-3.843587C5.947696-3.843587 6.031382-3.843587 6.031382-3.987049C6.031382-4.046824 5.989539-4.08269 5.929763-4.08269C5.804234-4.08269 5.660772-4.064757 5.529265-4.064757S5.260274-4.05878 5.128767-4.05878C4.991283-4.05878 4.853798-4.064757 4.716314-4.064757C4.584807-4.064757 4.435367-4.08269 4.303861-4.08269C4.267995-4.08269 4.166376-4.08269 4.166376-3.93325C4.166376-3.843587 4.25604-3.843587 4.363636-3.843587S4.620672-3.843587 4.746202-3.801743C4.746202-3.741968 4.746202-3.706102 4.722291-3.610461L4.375592-2.241594H2.259527L2.600249-3.598506C2.64807-3.795766 2.660025-3.843587 3.138232-3.843587C3.245828-3.843587 3.329514-3.843587 3.329514-3.987049C3.329514-4.046824 3.287671-4.08269 3.227895-4.08269C3.102366-4.08269 2.958904-4.064757 2.827397-4.064757S2.558406-4.05878 2.426899-4.05878C2.289415-4.05878 2.15193-4.064757 2.014446-4.064757C1.882939-4.064757 1.733499-4.08269 1.601993-4.08269C1.566127-4.08269 1.464508-4.08269 1.464508-3.93325C1.464508-3.843587 1.554172-3.843587 1.661768-3.843587S1.918804-3.843587 2.044334-3.801743C2.044334-3.741968 2.044334-3.706102 2.020423-3.610461L1.23736-.496139C1.183562-.280946 1.171606-.239103 .723288-.239103C.591781-.239103 .579826-.239103 .555915-.221171C.526027-.19726 .502117-.119552 .502117-.083686C.508095-.059776 .514072 0 .609714 0C.735243 0 .878705-.017933 1.010212-.017933C1.147696-.017933 1.279203-.02391 1.41071-.02391C1.548194-.02391 1.685679-.017933 1.823163-.017933C1.95467-.017933 2.10411 0 2.235616 0C2.271482 0 2.313325 0 2.337235-.041843C2.349191-.059776 2.373101-.131507 2.373101-.155417C2.355168-.239103 2.30137-.239103 2.163885-.239103C2.092154-.239103 2.002491-.245081 1.936737-.251059C1.829141-.263014 1.787298-.263014 1.787298-.328767C1.787298-.352677 1.793275-.376588 1.811208-.436364L2.199751-2.002491H4.315816L3.915318-.394521C3.867497-.280946 3.849564-.245081 3.38929-.239103C3.287671-.239103 3.275716-.239103 3.251806-.215193S3.203985-.113574 3.203985-.083686C3.209963-.059776 3.21594 0 3.311582 0C3.437111 0 3.580573-.017933 3.71208-.017933C3.849564-.017933 3.981071-.02391 4.112578-.02391C4.250062-.02391 4.387547-.017933 4.525031-.017933C4.656538-.017933 4.805978 0 4.937484 0C4.97335 0 5.015193 0 5.039103-.041843C5.051059-.059776 5.074969-.131507 5.074969-.155417C5.057036-.239103 5.003238-.239103 4.865753-.239103C4.794022-.239103 4.704359-.245081 4.638605-.251059C4.531009-.263014 4.489166-.263014 4.489166-.328767C4.489166-.352677 4.495143-.376588 4.513076-.436364L5.302117-3.598506Z'/> +<path id='g3-108' d='M1.793275-3.969116C1.799253-3.993026 1.811208-4.028892 1.811208-4.05878C1.811208-4.154421 1.691656-4.148443 1.619925-4.142466L.950436-4.088667C.848817-4.08269 .771108-4.076712 .771108-3.93325C.771108-3.843587 .848817-3.843587 .944458-3.843587C1.117808-3.843587 1.159651-3.825654 1.23736-3.801743C1.23736-3.730012 1.23736-3.718057 1.21345-3.622416L.484184-.71731C.466252-.651557 .454296-.597758 .454296-.514072C.454296-.14944 .789041 .059776 1.123786 .059776C1.374844 .059776 1.518306-.101619 1.601993-.239103C1.763387-.496139 1.841096-.836862 1.841096-.860772C1.841096-.872727 1.835118-.944458 1.727522-.944458C1.637858-.944458 1.625903-.902615 1.601993-.806974C1.494396-.382565 1.356912-.137484 1.147696-.137484C.926526-.137484 .926526-.376588 .926526-.436364C.926526-.532005 .938481-.579826 .956413-.645579L1.793275-3.969116Z'/> +<path id='g0-0' d='M4.758157-1.338979C4.853798-1.338979 5.003238-1.338979 5.003238-1.494396S4.853798-1.649813 4.758157-1.649813H.992279C.896638-1.649813 .747198-1.649813 .747198-1.494396S.896638-1.338979 .992279-1.338979H4.758157Z'/> +<path id='g6-49' d='M2.145953-3.795766C2.145953-3.975093 2.122042-3.975093 1.942715-3.975093C1.548194-3.592528 .938481-3.592528 .723288-3.592528V-3.359402C.878705-3.359402 1.273225-3.359402 1.63188-3.526775V-.508095C1.63188-.310834 1.63188-.233126 1.016189-.233126H.759153V0C1.08792-.02391 1.554172-.02391 1.888917-.02391S2.689913-.02391 3.01868 0V-.233126H2.761644C2.145953-.233126 2.145953-.310834 2.145953-.508095V-3.795766Z'/> +<path id='g6-91' d='M1.865006-4.172354V-4.483188H.89066V1.494396H1.865006V1.183562H1.201494V-4.172354H1.865006Z'/> +<path id='g6-93' d='M1.207472-4.483188H.233126V-4.172354H.896638V1.183562H.233126V1.494396H1.207472V-4.483188Z'/> +<path id='g1-0' d='M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z'/> +<path id='g2-98' d='M4.210908 2.727272C4.374544 2.727272 4.603635 2.727272 4.603635 2.50909S4.374544 2.290908 4.210908 2.290908H2.334545V-7.789089C2.334545-7.985452 2.334545-8.181816 2.116363-8.181816S1.898181-7.952725 1.898181-7.789089V2.334545C1.898181 2.683636 1.941818 2.727272 2.290908 2.727272H4.210908Z'/> +<path id='g2-99' d='M2.934545-7.789089C2.934545-7.985452 2.934545-8.181816 2.716363-8.181816S2.498181-7.952725 2.498181-7.789089V2.290908H.621818C.458182 2.290908 .229091 2.290908 .229091 2.50909S.458182 2.727272 .621818 2.727272H2.552727C2.901817 2.727272 2.934545 2.694545 2.934545 2.334545V-7.789089Z'/> +<path id='g8-43' d='M4.461817-2.50909H7.505452C7.65818-2.50909 7.865452-2.50909 7.865452-2.727272S7.65818-2.945454 7.505452-2.945454H4.461817V-5.999998C4.461817-6.152726 4.461817-6.359998 4.243635-6.359998S4.025453-6.152726 4.025453-5.999998V-2.945454H.970909C.818182-2.945454 .610909-2.945454 .610909-2.727272S.818182-2.50909 .970909-2.50909H4.025453V.545454C4.025453 .698182 4.025453 .905454 4.243635 .905454S4.461817 .698182 4.461817 .545454V-2.50909Z'/> +<path id='g8-49' d='M3.207272-6.981816C3.207272-7.243634 3.207272-7.265453 2.956363-7.265453C2.279999-6.567271 1.32-6.567271 .970909-6.567271V-6.229089C1.189091-6.229089 1.832727-6.229089 2.399999-6.512725V-.861818C2.399999-.469091 2.367272-.338182 1.385454-.338182H1.036363V0C1.418181-.032727 2.367272-.032727 2.803636-.032727S4.18909-.032727 4.570908 0V-.338182H4.221817C3.239999-.338182 3.207272-.458182 3.207272-.861818V-6.981816Z'/> +<path id='g8-61' d='M7.494543-3.567272C7.65818-3.567272 7.865452-3.567272 7.865452-3.785453S7.65818-4.003635 7.505452-4.003635H.970909C.818182-4.003635 .610909-4.003635 .610909-3.785453S.818182-3.567272 .981818-3.567272H7.494543ZM7.505452-1.450909C7.65818-1.450909 7.865452-1.450909 7.865452-1.66909S7.65818-1.887272 7.494543-1.887272H.981818C.818182-1.887272 .610909-1.887272 .610909-1.66909S.818182-1.450909 .970909-1.450909H7.505452Z'/> +<path id='g4-72' d='M6.36812-4.798007C6.447821-5.108842 6.463761-5.180573 7.021669-5.180573C7.204981-5.180573 7.300623-5.180573 7.300623-5.332005C7.300623-5.339975 7.292653-5.443587 7.165131-5.443587C7.013699-5.443587 6.822416-5.427646 6.670984-5.419676H6.168867C5.403736-5.419676 5.196513-5.443587 5.140722-5.443587C5.108842-5.443587 4.98132-5.443587 4.98132-5.292154C4.98132-5.180573 5.084932-5.180573 5.212453-5.180573C5.475467-5.180573 5.7066-5.180573 5.7066-5.053051C5.7066-5.021171 5.69863-5.0132 5.67472-4.909589L5.196513-2.964882H2.630137L3.092403-4.821918C3.164134-5.108842 3.180075-5.180573 3.761893-5.180573C3.921295-5.180573 4.016936-5.180573 4.016936-5.332005C4.016936-5.339975 4.008966-5.443587 3.881445-5.443587C3.730012-5.443587 3.53873-5.427646 3.387298-5.419676H2.885181C2.12005-5.419676 1.912827-5.443587 1.857036-5.443587C1.825156-5.443587 1.697634-5.443587 1.697634-5.292154C1.697634-5.180573 1.801245-5.180573 1.928767-5.180573C2.191781-5.180573 2.422914-5.180573 2.422914-5.053051C2.422914-5.021171 2.414944-5.0132 2.391034-4.909589L1.323039-.629639C1.251308-.326775 1.235367-.263014 .645579-.263014C.494147-.263014 .398506-.263014 .398506-.111582C.398506-.079701 .422416 0 .526027 0C.67746 0 .868742-.01594 1.020174-.02391H1.530262C2.295392-.02391 2.502615 0 2.558406 0C2.606227 0 2.717808 0 2.717808-.151432C2.717808-.263014 2.614197-.263014 2.478705-.263014C2.454795-.263014 2.311333-.263014 2.175841-.278954C2.008468-.294894 1.992528-.318804 1.992528-.390535C1.992528-.430386 2.008468-.478207 2.016438-.518057L2.558406-2.701868H5.124782C4.893649-1.761395 4.582814-.518057 4.574844-.518057C4.503113-.270984 4.359651-.270984 3.881445-.263014C3.785803-.263014 3.682192-.263014 3.682192-.111582C3.682192-.079701 3.706102 0 3.809714 0C3.961146 0 4.152428-.01594 4.303861-.02391H4.813948C5.579078-.02391 5.786301 0 5.842092 0C5.889913 0 6.001494 0 6.001494-.151432C6.001494-.263014 5.897883-.263014 5.762391-.263014C5.738481-.263014 5.595019-.263014 5.459527-.278954C5.292154-.294894 5.276214-.318804 5.276214-.390535C5.276214-.430386 5.292154-.478207 5.300125-.518057L6.36812-4.798007Z'/> +<path id='g4-102' d='M3.052553-3.172105H3.793773C3.953176-3.172105 4.048817-3.172105 4.048817-3.323537C4.048817-3.435118 3.945205-3.435118 3.809714-3.435118H3.100374C3.227895-4.152428 3.307597-4.606725 3.387298-4.96538C3.419178-5.100872 3.443088-5.188543 3.56264-5.284184C3.666252-5.371856 3.730012-5.387796 3.817684-5.387796C3.937235-5.387796 4.064757-5.363885 4.168369-5.300125C4.128518-5.284184 4.080697-5.260274 4.040847-5.236364C3.905355-5.164633 3.809714-5.021171 3.809714-4.861768C3.809714-4.678456 3.953176-4.566874 4.128518-4.566874C4.359651-4.566874 4.574844-4.766127 4.574844-5.045081C4.574844-5.419676 4.192279-5.610959 3.809714-5.610959C3.53873-5.610959 3.036613-5.483437 2.781569-4.750187C2.709838-4.566874 2.709838-4.550934 2.494645-3.435118H1.896887C1.737484-3.435118 1.641843-3.435118 1.641843-3.283686C1.641843-3.172105 1.745455-3.172105 1.880946-3.172105H2.446824L1.872976-.079701C1.721544 .72528 1.601993 1.40274 1.179577 1.40274C1.155666 1.40274 .988294 1.40274 .836862 1.307098C1.203487 1.219427 1.203487 .884682 1.203487 .876712C1.203487 .6934 1.060025 .581818 .884682 .581818C.669489 .581818 .438356 .765131 .438356 1.067995C.438356 1.40274 .781071 1.625903 1.179577 1.625903C1.665753 1.625903 2.000498 1.115816 2.10411 .916563C2.391034 .390535 2.574346-.605729 2.590286-.68543L3.052553-3.172105Z'/> +<path id='g4-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g4-110' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.833126-2.279452 1.833126-2.287422 2.016438-2.550436C2.279452-2.940971 2.654047-3.291656 3.188045-3.291656C3.474969-3.291656 3.642341-3.124284 3.642341-2.749689C3.642341-2.311333 3.307597-1.40274 3.156164-1.012204C3.052553-.749191 3.052553-.70137 3.052553-.597758C3.052553-.143462 3.427148 .079701 3.769863 .079701C4.550934 .079701 4.877709-1.036115 4.877709-1.139726C4.877709-1.219427 4.813948-1.243337 4.758157-1.243337C4.662516-1.243337 4.646575-1.187547 4.622665-1.107846C4.431382-.454296 4.096638-.143462 3.793773-.143462C3.666252-.143462 3.602491-.223163 3.602491-.406476S3.666252-.765131 3.745953-.964384C3.865504-1.267248 4.216189-2.183811 4.216189-2.630137C4.216189-3.227895 3.801743-3.514819 3.227895-3.514819C2.582316-3.514819 2.16787-3.124284 1.936737-2.82142C1.880946-3.259776 1.530262-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.318804-2.709838 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g4-112' d='M.414446 .964384C.350685 1.219427 .334745 1.283188 .01594 1.283188C-.095641 1.283188-.191283 1.283188-.191283 1.43462C-.191283 1.506351-.119552 1.546202-.079701 1.546202C0 1.546202 .03188 1.522291 .621669 1.522291C1.195517 1.522291 1.362889 1.546202 1.41868 1.546202C1.45056 1.546202 1.570112 1.546202 1.570112 1.39477C1.570112 1.283188 1.458531 1.283188 1.362889 1.283188C.980324 1.283188 .980324 1.235367 .980324 1.163636C.980324 1.107846 1.123786 .541968 1.362889-.390535C1.466501-.207223 1.713574 .079701 2.14396 .079701C3.124284 .079701 4.144458-1.052055 4.144458-2.207721C4.144458-2.996762 3.634371-3.514819 2.996762-3.514819C2.518555-3.514819 2.13599-3.188045 1.904857-2.948941C1.737484-3.514819 1.203487-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.326775-2.725778 .239103-2.319303 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.629639-2.83736 .773101-3.291656 1.099875-3.291656C1.299128-3.291656 1.354919-3.108344 1.354919-2.917061C1.354919-2.83736 1.323039-2.646077 1.307098-2.582316L.414446 .964384ZM1.880946-2.454795C1.920797-2.590286 1.920797-2.606227 2.040349-2.749689C2.343213-3.108344 2.685928-3.291656 2.972852-3.291656C3.371357-3.291656 3.52279-2.901121 3.52279-2.542466C3.52279-2.247572 3.347447-1.39477 3.108344-.924533C2.901121-.494147 2.518555-.143462 2.14396-.143462C1.601993-.143462 1.474471-.765131 1.474471-.820922C1.474471-.836862 1.490411-.924533 1.498381-.948443L1.880946-2.454795Z'/> +<path id='g4-115' d='M3.211955-2.996762C3.028643-2.964882 2.86127-2.82142 2.86127-2.622167C2.86127-2.478705 2.956912-2.375093 3.132254-2.375093C3.251806-2.375093 3.498879-2.462765 3.498879-2.82142C3.498879-3.315567 2.980822-3.514819 2.486675-3.514819C1.41868-3.514819 1.083935-2.757659 1.083935-2.351183C1.083935-2.271482 1.083935-1.984558 1.378829-1.761395C1.562142-1.617933 1.697634-1.594022 2.11208-1.514321C2.391034-1.458531 2.84533-1.378829 2.84533-.964384C2.84533-.757161 2.693898-.494147 2.470735-.342715C2.175841-.151432 1.785305-.143462 1.657783-.143462C1.466501-.143462 .924533-.175342 .72528-.494147C1.131756-.510087 1.187547-.836862 1.187547-.932503C1.187547-1.171606 .972354-1.227397 .876712-1.227397C.749191-1.227397 .422416-1.131756 .422416-.6934C.422416-.223163 .916563 .079701 1.657783 .079701C3.044583 .079701 3.339477-.900623 3.339477-1.235367C3.339477-1.952677 2.558406-2.10411 2.263512-2.1599C1.880946-2.231631 1.570112-2.287422 1.570112-2.622167C1.570112-2.765629 1.705604-3.291656 2.478705-3.291656C2.781569-3.291656 3.092403-3.203985 3.211955-2.996762Z'/> +<path id='g7-43' d='M3.474969-1.809215H5.818182C5.929763-1.809215 6.105106-1.809215 6.105106-1.992528S5.929763-2.175841 5.818182-2.175841H3.474969V-4.527024C3.474969-4.638605 3.474969-4.813948 3.291656-4.813948S3.108344-4.638605 3.108344-4.527024V-2.175841H.757161C.645579-2.175841 .470237-2.175841 .470237-1.992528S.645579-1.809215 .757161-1.809215H3.108344V.541968C3.108344 .653549 3.108344 .828892 3.291656 .828892S3.474969 .653549 3.474969 .541968V-1.809215Z'/> +<path id='g7-50' d='M2.247572-1.625903C2.375093-1.745455 2.709838-2.008468 2.83736-2.12005C3.331507-2.574346 3.801743-3.012702 3.801743-3.737983C3.801743-4.686426 3.004732-5.300125 2.008468-5.300125C1.052055-5.300125 .422416-4.574844 .422416-3.865504C.422416-3.474969 .73325-3.419178 .844832-3.419178C1.012204-3.419178 1.259278-3.53873 1.259278-3.841594C1.259278-4.25604 .860772-4.25604 .765131-4.25604C.996264-4.837858 1.530262-5.037111 1.920797-5.037111C2.662017-5.037111 3.044583-4.407472 3.044583-3.737983C3.044583-2.909091 2.462765-2.303362 1.522291-1.338979L.518057-.302864C.422416-.215193 .422416-.199253 .422416 0H3.57061L3.801743-1.42665H3.55467C3.53076-1.267248 3.466999-.868742 3.371357-.71731C3.323537-.653549 2.717808-.653549 2.590286-.653549H1.171606L2.247572-1.625903Z'/> +<path id='g7-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g7-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g5-110' d='M.96-.643636C.927272-.48 .861818-.229091 .861818-.174545C.861818 .021818 1.014545 .12 1.178181 .12C1.309091 .12 1.505454 .032727 1.581818-.185454C1.592727-.207273 1.723636-.72 1.78909-.992727L2.02909-1.974545C2.094545-2.214545 2.159999-2.454545 2.214545-2.705454C2.258181-2.890908 2.345454-3.207272 2.356363-3.250908C2.519999-3.58909 3.098181-4.581817 4.134544-4.581817C4.625453-4.581817 4.723635-4.178181 4.723635-3.818181C4.723635-3.141817 4.18909-1.745454 4.014544-1.276363C3.916363-1.025454 3.905453-.894545 3.905453-.774545C3.905453-.261818 4.287272 .12 4.799999 .12C5.825453 .12 6.229089-1.472727 6.229089-1.56C6.229089-1.66909 6.130907-1.66909 6.09818-1.66909C5.989089-1.66909 5.989089-1.636363 5.934544-1.472727C5.716362-.730909 5.356362-.12 4.821817-.12C4.636362-.12 4.559999-.229091 4.559999-.48C4.559999-.752727 4.658181-1.014545 4.756362-1.254545C4.963635-1.832727 5.421817-3.032726 5.421817-3.654544C5.421817-4.385453 4.952726-4.821817 4.167272-4.821817C3.185454-4.821817 2.650908-4.123635 2.465454-3.872726C2.410908-4.483635 1.963636-4.821817 1.461818-4.821817S.752727-4.396362 .643636-4.199999C.469091-3.82909 .316364-3.185454 .316364-3.141817C.316364-3.032726 .425454-3.032726 .447273-3.032726C.556363-3.032726 .567273-3.043636 .632727-3.283635C.818182-4.058181 1.036363-4.581817 1.429091-4.581817C1.647272-4.581817 1.767272-4.439999 1.767272-4.079999C1.767272-3.850908 1.734545-3.730908 1.592727-3.163635L.96-.643636Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -65.54)'> +<use x='70.734745' y='72.661996' xlink:href='#g5-110'/> +<use x='77.282761' y='66.911907' xlink:href='#g7-91'/> +<use x='79.635085' y='66.911907' xlink:href='#g4-108'/> +<use x='82.25721' y='66.911907' xlink:href='#g7-93'/> +<use x='77.282761' y='76.096079' xlink:href='#g4-72'/> +<use x='88.381445' y='72.661996' xlink:href='#g8-61'/> +<use x='99.896569' y='72.661996' xlink:href='#g2-98'/> +<use x='105.940579' y='66.506156' xlink:href='#g4-110'/> +<use x='111.078781' y='62.266501' xlink:href='#g6-91'/> +<use x='113.18198' y='62.266501' xlink:href='#g3-108'/> +<use x='115.582708' y='62.266501' xlink:href='#g0-0'/> +<use x='121.338861' y='62.266501' xlink:href='#g6-49'/> +<use x='124.991787' y='62.266501' xlink:href='#g6-93'/> +<use x='111.078781' y='69.280208' xlink:href='#g3-72'/> +<use x='127.593086' y='66.506156' xlink:href='#g7-43'/> +<use x='134.179592' y='66.506156' xlink:href='#g7-50'/> +<use x='138.413775' y='66.506156' xlink:href='#g4-112'/> +<use x='142.676552' y='63.69337' xlink:href='#g6-91'/> +<use x='144.779751' y='63.69337' xlink:href='#g3-108'/> +<use x='147.18048' y='63.69337' xlink:href='#g6-93'/> +<use x='149.781805' y='66.506156' xlink:href='#g1-0'/> +<use x='156.368311' y='66.506156' xlink:href='#g4-102'/> +<use x='161.315078' y='63.69337' xlink:href='#g6-91'/> +<use x='163.418277' y='63.69337' xlink:href='#g3-108'/> +<use x='165.819005' y='63.69337' xlink:href='#g6-93'/> +<rect x='105.940579' y='69.71656' height='.436352' width='62.479751'/> +<use x='131.669869' y='77.42473' xlink:href='#g4-115'/> +<use x='135.585787' y='75.072452' xlink:href='#g6-91'/> +<use x='137.688986' y='75.072452' xlink:href='#g3-108'/> +<use x='140.089715' y='75.072452' xlink:href='#g6-93'/> +<use x='172.040042' y='72.661996' xlink:href='#g8-43'/> +<use x='182.949116' y='72.661996' xlink:href='#g8-49'/> +<use x='188.40368' y='72.661996' xlink:href='#g2-99'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_output_shape.svg b/_resources/layer_output_shape.svg new file mode 100644 index 0000000..538fcca --- /dev/null +++ b/_resources/layer_output_shape.svg @@ -0,0 +1,33 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='83.900426pt' height='17.511011pt' viewBox='-.239051 -.22797 83.900426 17.511011'> +<defs> +<path id='g0-2' d='M4.65056-3.323537L2.259527-5.702615C2.116065-5.846077 2.092154-5.869988 1.996513-5.869988C1.876961-5.869988 1.75741-5.762391 1.75741-5.630884C1.75741-5.547198 1.78132-5.523288 1.912827-5.391781L4.303861-2.988792L1.912827-.585803C1.78132-.454296 1.75741-.430386 1.75741-.3467C1.75741-.215193 1.876961-.107597 1.996513-.107597C2.092154-.107597 2.116065-.131507 2.259527-.274969L4.638605-2.654047L7.113325-.179328C7.137235-.167372 7.220922-.107597 7.292653-.107597C7.436115-.107597 7.531756-.215193 7.531756-.3467C7.531756-.37061 7.531756-.418431 7.49589-.478207C7.483935-.502117 5.583064-2.379078 4.985305-2.988792L7.173101-5.176588C7.232877-5.248319 7.412204-5.403736 7.47198-5.475467C7.483935-5.499377 7.531756-5.547198 7.531756-5.630884C7.531756-5.762391 7.436115-5.869988 7.292653-5.869988C7.197011-5.869988 7.149191-5.822167 7.017684-5.69066L4.65056-3.323537Z'/> +<path id='g1-72' d='M6.36812-4.798007C6.447821-5.108842 6.463761-5.180573 7.021669-5.180573C7.204981-5.180573 7.300623-5.180573 7.300623-5.332005C7.300623-5.339975 7.292653-5.443587 7.165131-5.443587C7.013699-5.443587 6.822416-5.427646 6.670984-5.419676H6.168867C5.403736-5.419676 5.196513-5.443587 5.140722-5.443587C5.108842-5.443587 4.98132-5.443587 4.98132-5.292154C4.98132-5.180573 5.084932-5.180573 5.212453-5.180573C5.475467-5.180573 5.7066-5.180573 5.7066-5.053051C5.7066-5.021171 5.69863-5.0132 5.67472-4.909589L5.196513-2.964882H2.630137L3.092403-4.821918C3.164134-5.108842 3.180075-5.180573 3.761893-5.180573C3.921295-5.180573 4.016936-5.180573 4.016936-5.332005C4.016936-5.339975 4.008966-5.443587 3.881445-5.443587C3.730012-5.443587 3.53873-5.427646 3.387298-5.419676H2.885181C2.12005-5.419676 1.912827-5.443587 1.857036-5.443587C1.825156-5.443587 1.697634-5.443587 1.697634-5.292154C1.697634-5.180573 1.801245-5.180573 1.928767-5.180573C2.191781-5.180573 2.422914-5.180573 2.422914-5.053051C2.422914-5.021171 2.414944-5.0132 2.391034-4.909589L1.323039-.629639C1.251308-.326775 1.235367-.263014 .645579-.263014C.494147-.263014 .398506-.263014 .398506-.111582C.398506-.079701 .422416 0 .526027 0C.67746 0 .868742-.01594 1.020174-.02391H1.530262C2.295392-.02391 2.502615 0 2.558406 0C2.606227 0 2.717808 0 2.717808-.151432C2.717808-.263014 2.614197-.263014 2.478705-.263014C2.454795-.263014 2.311333-.263014 2.175841-.278954C2.008468-.294894 1.992528-.318804 1.992528-.390535C1.992528-.430386 2.008468-.478207 2.016438-.518057L2.558406-2.701868H5.124782C4.893649-1.761395 4.582814-.518057 4.574844-.518057C4.503113-.270984 4.359651-.270984 3.881445-.263014C3.785803-.263014 3.682192-.263014 3.682192-.111582C3.682192-.079701 3.706102 0 3.809714 0C3.961146 0 4.152428-.01594 4.303861-.02391H4.813948C5.579078-.02391 5.786301 0 5.842092 0C5.889913 0 6.001494 0 6.001494-.151432C6.001494-.263014 5.897883-.263014 5.762391-.263014C5.738481-.263014 5.595019-.263014 5.459527-.278954C5.292154-.294894 5.276214-.318804 5.276214-.390535C5.276214-.430386 5.292154-.478207 5.300125-.518057L6.36812-4.798007Z'/> +<path id='g1-87' d='M7.7868-4.487173C8.145455-5.084932 8.416438-5.156663 8.679452-5.180573C8.751183-5.188543 8.846824-5.196513 8.846824-5.332005C8.846824-5.387796 8.799004-5.443587 8.735243-5.443587C8.599751-5.443587 8.727273-5.419676 8.097634-5.419676C7.571606-5.419676 7.340473-5.443587 7.300623-5.443587C7.173101-5.443587 7.149191-5.355915 7.149191-5.292154C7.149191-5.188543 7.260772-5.180573 7.284682-5.180573C7.507846-5.172603 7.707098-5.092902 7.707098-4.893649C7.707098-4.790037 7.627397-4.654545 7.587547-4.590785L5.379826-.876712L5.00523-4.837858C4.99726-4.853798 4.98929-4.909589 4.98929-4.933499C4.98929-5.053051 5.140722-5.180573 5.531258-5.180573C5.650809-5.180573 5.746451-5.180573 5.746451-5.332005C5.746451-5.395766 5.69863-5.443587 5.618929-5.443587S5.244334-5.427646 5.164633-5.419676H4.686426C3.977086-5.419676 3.889415-5.443587 3.825654-5.443587C3.793773-5.443587 3.666252-5.443587 3.666252-5.292154C3.666252-5.180573 3.769863-5.180573 3.889415-5.180573C4.224159-5.180573 4.24807-5.116812 4.26401-5.00523C4.26401-4.98929 4.311831-4.527024 4.311831-4.503113S4.311831-4.439352 4.25604-4.351681L2.183811-.876712L1.809215-4.798007C1.809215-4.837858 1.801245-4.901619 1.801245-4.941469C1.801245-5.00523 1.865006-5.180573 2.351183-5.180573C2.438854-5.180573 2.550436-5.180573 2.550436-5.332005C2.550436-5.395766 2.494645-5.443587 2.422914-5.443587C2.343213-5.443587 2.048319-5.427646 1.968618-5.419676H1.490411C.757161-5.419676 .72528-5.443587 .629639-5.443587C.502117-5.443587 .478207-5.355915 .478207-5.292154C.478207-5.180573 .589788-5.180573 .67746-5.180573C1.052055-5.180573 1.060025-5.124782 1.075965-4.925529L1.538232-.063761C1.554172 .087671 1.562142 .167372 1.697634 .167372C1.761395 .167372 1.841096 .143462 1.928767 0L4.351681-4.080697L4.734247-.055791C4.750187 .087671 4.758157 .167372 4.885679 .167372C5.021171 .167372 5.068991 .079701 5.116812 0L7.7868-4.487173Z'/> +<path id='g1-99' d='M3.259776-3.052553C2.933001-3.012702 2.82939-2.765629 2.82939-2.606227C2.82939-2.375093 3.036613-2.311333 3.140224-2.311333C3.180075-2.311333 3.58655-2.343213 3.58655-2.82939S3.060523-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.541968 .868742 .079701 1.753425 .079701C3.012702 .079701 3.674222-.72528 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.769365-.143462C1.338979-.143462 .996264-.406476 .996264-1.012204C.996264-1.362889 1.155666-2.207721 1.530262-2.701868C1.880946-3.148194 2.279452-3.291656 2.590286-3.291656C2.685928-3.291656 3.052553-3.283686 3.259776-3.052553Z'/> +<path id='g1-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g3-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g3-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g2-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -61.02)'> +<use x='56.413267' y='65.886243' xlink:href='#g2-110'/> +<use x='63.400873' y='59.775841' xlink:href='#g3-91'/> +<use x='65.753197' y='59.775841' xlink:href='#g1-108'/> +<use x='68.375322' y='59.775841' xlink:href='#g3-93'/> +<use x='63.400873' y='69.127354' xlink:href='#g1-72'/> +<use x='74.125974' y='65.886243' xlink:href='#g0-2'/> +<use x='86.081134' y='65.886243' xlink:href='#g2-110'/> +<use x='93.06874' y='59.775841' xlink:href='#g3-91'/> +<use x='95.421064' y='59.775841' xlink:href='#g1-108'/> +<use x='98.043189' y='59.775841' xlink:href='#g3-93'/> +<use x='93.06874' y='69.127354' xlink:href='#g1-87'/> +<use x='105.389298' y='65.886243' xlink:href='#g0-2'/> +<use x='117.344458' y='65.886243' xlink:href='#g2-110'/> +<use x='124.332064' y='60.950057' xlink:href='#g3-91'/> +<use x='126.684388' y='60.950057' xlink:href='#g1-108'/> +<use x='129.306513' y='60.950057' xlink:href='#g3-93'/> +<use x='124.332064' y='68.841758' xlink:href='#g1-99'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_output_width.svg b/_resources/layer_output_width.svg new file mode 100644 index 0000000..0233dd9 --- /dev/null +++ b/_resources/layer_output_width.svg @@ -0,0 +1,64 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='138.0848pt' height='22.284864pt' viewBox='-.299738 -.244857 138.0848 22.284864'> +<defs> +<path id='g3-87' d='M6.587298-3.36538C6.832379-3.7599 7.047572-3.825654 7.268742-3.843587C7.340473-3.849564 7.418182-3.855542 7.418182-3.993026C7.418182-4.040847 7.382316-4.08269 7.328518-4.08269C7.197011-4.08269 6.999751-4.05878 6.850311-4.05878S6.318306-4.08269 6.168867-4.08269C6.133001-4.08269 6.03736-4.08269 6.03736-3.93325C6.03736-3.849564 6.115068-3.843587 6.162889-3.843587C6.324284-3.837609 6.485679-3.783811 6.485679-3.646326C6.485679-3.568618 6.43188-3.484932 6.396015-3.431133L4.662516-.669489L4.285928-3.694147C4.285928-3.730012 4.315816-3.837609 4.716314-3.843587C4.782067-3.843587 4.871731-3.843587 4.871731-3.993026C4.871731-4.040847 4.835866-4.08269 4.770112-4.08269C4.656538-4.08269 4.519054-4.064757 4.399502-4.064757C4.273973-4.064757 4.112578-4.05878 3.987049-4.05878C3.873474-4.05878 3.771856-4.064757 3.658281-4.064757C3.556663-4.064757 3.431133-4.08269 3.329514-4.08269C3.293649-4.08269 3.198007-4.08269 3.198007-3.93325C3.198007-3.843587 3.281694-3.843587 3.353425-3.843587C3.461021-3.843587 3.634371-3.843587 3.670237-3.777833C3.682192-3.741968 3.724035-3.443088 3.724035-3.401245C3.724035-3.38929 3.724035-3.377335 3.676214-3.311582L2.026401-.669489L1.643836-3.700125C1.673724-3.7599 1.715567-3.843587 2.068244-3.843587C2.145953-3.843587 2.235616-3.843587 2.235616-3.993026C2.235616-4.022914 2.217684-4.08269 2.133998-4.08269C2.020423-4.08269 1.882939-4.064757 1.763387-4.064757C1.637858-4.064757 1.476463-4.05878 1.350934-4.05878C1.23736-4.05878 1.135741-4.064757 1.022167-4.064757C.91457-4.064757 .789041-4.08269 .687422-4.08269C.561893-4.08269 .555915-3.957161 .555915-3.927273C.573848-3.843587 .633624-3.843587 .735243-3.843587C1.028144-3.843587 1.034122-3.813699 1.052055-3.652304L1.506351-.041843C1.524284 .077709 1.536239 .125529 1.649813 .125529C1.751432 .125529 1.799253 .077709 1.853051-.011955L3.765878-3.072478L4.148443-.041843C4.166376 .077709 4.172354 .125529 4.285928 .125529C4.387547 .125529 4.42939 .083686 4.483188 0L6.587298-3.36538Z'/> +<path id='g3-108' d='M1.793275-3.969116C1.799253-3.993026 1.811208-4.028892 1.811208-4.05878C1.811208-4.154421 1.691656-4.148443 1.619925-4.142466L.950436-4.088667C.848817-4.08269 .771108-4.076712 .771108-3.93325C.771108-3.843587 .848817-3.843587 .944458-3.843587C1.117808-3.843587 1.159651-3.825654 1.23736-3.801743C1.23736-3.730012 1.23736-3.718057 1.21345-3.622416L.484184-.71731C.466252-.651557 .454296-.597758 .454296-.514072C.454296-.14944 .789041 .059776 1.123786 .059776C1.374844 .059776 1.518306-.101619 1.601993-.239103C1.763387-.496139 1.841096-.836862 1.841096-.860772C1.841096-.872727 1.835118-.944458 1.727522-.944458C1.637858-.944458 1.625903-.902615 1.601993-.806974C1.494396-.382565 1.356912-.137484 1.147696-.137484C.926526-.137484 .926526-.376588 .926526-.436364C.926526-.532005 .938481-.579826 .956413-.645579L1.793275-3.969116Z'/> +<path id='g0-0' d='M4.758157-1.338979C4.853798-1.338979 5.003238-1.338979 5.003238-1.494396S4.853798-1.649813 4.758157-1.649813H.992279C.896638-1.649813 .747198-1.649813 .747198-1.494396S.896638-1.338979 .992279-1.338979H4.758157Z'/> +<path id='g6-49' d='M2.145953-3.795766C2.145953-3.975093 2.122042-3.975093 1.942715-3.975093C1.548194-3.592528 .938481-3.592528 .723288-3.592528V-3.359402C.878705-3.359402 1.273225-3.359402 1.63188-3.526775V-.508095C1.63188-.310834 1.63188-.233126 1.016189-.233126H.759153V0C1.08792-.02391 1.554172-.02391 1.888917-.02391S2.689913-.02391 3.01868 0V-.233126H2.761644C2.145953-.233126 2.145953-.310834 2.145953-.508095V-3.795766Z'/> +<path id='g6-91' d='M1.865006-4.172354V-4.483188H.89066V1.494396H1.865006V1.183562H1.201494V-4.172354H1.865006Z'/> +<path id='g6-93' d='M1.207472-4.483188H.233126V-4.172354H.896638V1.183562H.233126V1.494396H1.207472V-4.483188Z'/> +<path id='g1-0' d='M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z'/> +<path id='g2-98' d='M4.210908 2.727272C4.374544 2.727272 4.603635 2.727272 4.603635 2.50909S4.374544 2.290908 4.210908 2.290908H2.334545V-7.789089C2.334545-7.985452 2.334545-8.181816 2.116363-8.181816S1.898181-7.952725 1.898181-7.789089V2.334545C1.898181 2.683636 1.941818 2.727272 2.290908 2.727272H4.210908Z'/> +<path id='g2-99' d='M2.934545-7.789089C2.934545-7.985452 2.934545-8.181816 2.716363-8.181816S2.498181-7.952725 2.498181-7.789089V2.290908H.621818C.458182 2.290908 .229091 2.290908 .229091 2.50909S.458182 2.727272 .621818 2.727272H2.552727C2.901817 2.727272 2.934545 2.694545 2.934545 2.334545V-7.789089Z'/> +<path id='g8-43' d='M4.461817-2.50909H7.505452C7.65818-2.50909 7.865452-2.50909 7.865452-2.727272S7.65818-2.945454 7.505452-2.945454H4.461817V-5.999998C4.461817-6.152726 4.461817-6.359998 4.243635-6.359998S4.025453-6.152726 4.025453-5.999998V-2.945454H.970909C.818182-2.945454 .610909-2.945454 .610909-2.727272S.818182-2.50909 .970909-2.50909H4.025453V.545454C4.025453 .698182 4.025453 .905454 4.243635 .905454S4.461817 .698182 4.461817 .545454V-2.50909Z'/> +<path id='g8-49' d='M3.207272-6.981816C3.207272-7.243634 3.207272-7.265453 2.956363-7.265453C2.279999-6.567271 1.32-6.567271 .970909-6.567271V-6.229089C1.189091-6.229089 1.832727-6.229089 2.399999-6.512725V-.861818C2.399999-.469091 2.367272-.338182 1.385454-.338182H1.036363V0C1.418181-.032727 2.367272-.032727 2.803636-.032727S4.18909-.032727 4.570908 0V-.338182H4.221817C3.239999-.338182 3.207272-.458182 3.207272-.861818V-6.981816Z'/> +<path id='g8-61' d='M7.494543-3.567272C7.65818-3.567272 7.865452-3.567272 7.865452-3.785453S7.65818-4.003635 7.505452-4.003635H.970909C.818182-4.003635 .610909-4.003635 .610909-3.785453S.818182-3.567272 .981818-3.567272H7.494543ZM7.505452-1.450909C7.65818-1.450909 7.865452-1.450909 7.865452-1.66909S7.65818-1.887272 7.494543-1.887272H.981818C.818182-1.887272 .610909-1.887272 .610909-1.66909S.818182-1.450909 .970909-1.450909H7.505452Z'/> +<path id='g4-87' d='M7.7868-4.487173C8.145455-5.084932 8.416438-5.156663 8.679452-5.180573C8.751183-5.188543 8.846824-5.196513 8.846824-5.332005C8.846824-5.387796 8.799004-5.443587 8.735243-5.443587C8.599751-5.443587 8.727273-5.419676 8.097634-5.419676C7.571606-5.419676 7.340473-5.443587 7.300623-5.443587C7.173101-5.443587 7.149191-5.355915 7.149191-5.292154C7.149191-5.188543 7.260772-5.180573 7.284682-5.180573C7.507846-5.172603 7.707098-5.092902 7.707098-4.893649C7.707098-4.790037 7.627397-4.654545 7.587547-4.590785L5.379826-.876712L5.00523-4.837858C4.99726-4.853798 4.98929-4.909589 4.98929-4.933499C4.98929-5.053051 5.140722-5.180573 5.531258-5.180573C5.650809-5.180573 5.746451-5.180573 5.746451-5.332005C5.746451-5.395766 5.69863-5.443587 5.618929-5.443587S5.244334-5.427646 5.164633-5.419676H4.686426C3.977086-5.419676 3.889415-5.443587 3.825654-5.443587C3.793773-5.443587 3.666252-5.443587 3.666252-5.292154C3.666252-5.180573 3.769863-5.180573 3.889415-5.180573C4.224159-5.180573 4.24807-5.116812 4.26401-5.00523C4.26401-4.98929 4.311831-4.527024 4.311831-4.503113S4.311831-4.439352 4.25604-4.351681L2.183811-.876712L1.809215-4.798007C1.809215-4.837858 1.801245-4.901619 1.801245-4.941469C1.801245-5.00523 1.865006-5.180573 2.351183-5.180573C2.438854-5.180573 2.550436-5.180573 2.550436-5.332005C2.550436-5.395766 2.494645-5.443587 2.422914-5.443587C2.343213-5.443587 2.048319-5.427646 1.968618-5.419676H1.490411C.757161-5.419676 .72528-5.443587 .629639-5.443587C.502117-5.443587 .478207-5.355915 .478207-5.292154C.478207-5.180573 .589788-5.180573 .67746-5.180573C1.052055-5.180573 1.060025-5.124782 1.075965-4.925529L1.538232-.063761C1.554172 .087671 1.562142 .167372 1.697634 .167372C1.761395 .167372 1.841096 .143462 1.928767 0L4.351681-4.080697L4.734247-.055791C4.750187 .087671 4.758157 .167372 4.885679 .167372C5.021171 .167372 5.068991 .079701 5.116812 0L7.7868-4.487173Z'/> +<path id='g4-102' d='M3.052553-3.172105H3.793773C3.953176-3.172105 4.048817-3.172105 4.048817-3.323537C4.048817-3.435118 3.945205-3.435118 3.809714-3.435118H3.100374C3.227895-4.152428 3.307597-4.606725 3.387298-4.96538C3.419178-5.100872 3.443088-5.188543 3.56264-5.284184C3.666252-5.371856 3.730012-5.387796 3.817684-5.387796C3.937235-5.387796 4.064757-5.363885 4.168369-5.300125C4.128518-5.284184 4.080697-5.260274 4.040847-5.236364C3.905355-5.164633 3.809714-5.021171 3.809714-4.861768C3.809714-4.678456 3.953176-4.566874 4.128518-4.566874C4.359651-4.566874 4.574844-4.766127 4.574844-5.045081C4.574844-5.419676 4.192279-5.610959 3.809714-5.610959C3.53873-5.610959 3.036613-5.483437 2.781569-4.750187C2.709838-4.566874 2.709838-4.550934 2.494645-3.435118H1.896887C1.737484-3.435118 1.641843-3.435118 1.641843-3.283686C1.641843-3.172105 1.745455-3.172105 1.880946-3.172105H2.446824L1.872976-.079701C1.721544 .72528 1.601993 1.40274 1.179577 1.40274C1.155666 1.40274 .988294 1.40274 .836862 1.307098C1.203487 1.219427 1.203487 .884682 1.203487 .876712C1.203487 .6934 1.060025 .581818 .884682 .581818C.669489 .581818 .438356 .765131 .438356 1.067995C.438356 1.40274 .781071 1.625903 1.179577 1.625903C1.665753 1.625903 2.000498 1.115816 2.10411 .916563C2.391034 .390535 2.574346-.605729 2.590286-.68543L3.052553-3.172105Z'/> +<path id='g4-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g4-110' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.833126-2.279452 1.833126-2.287422 2.016438-2.550436C2.279452-2.940971 2.654047-3.291656 3.188045-3.291656C3.474969-3.291656 3.642341-3.124284 3.642341-2.749689C3.642341-2.311333 3.307597-1.40274 3.156164-1.012204C3.052553-.749191 3.052553-.70137 3.052553-.597758C3.052553-.143462 3.427148 .079701 3.769863 .079701C4.550934 .079701 4.877709-1.036115 4.877709-1.139726C4.877709-1.219427 4.813948-1.243337 4.758157-1.243337C4.662516-1.243337 4.646575-1.187547 4.622665-1.107846C4.431382-.454296 4.096638-.143462 3.793773-.143462C3.666252-.143462 3.602491-.223163 3.602491-.406476S3.666252-.765131 3.745953-.964384C3.865504-1.267248 4.216189-2.183811 4.216189-2.630137C4.216189-3.227895 3.801743-3.514819 3.227895-3.514819C2.582316-3.514819 2.16787-3.124284 1.936737-2.82142C1.880946-3.259776 1.530262-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.318804-2.709838 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g4-112' d='M.414446 .964384C.350685 1.219427 .334745 1.283188 .01594 1.283188C-.095641 1.283188-.191283 1.283188-.191283 1.43462C-.191283 1.506351-.119552 1.546202-.079701 1.546202C0 1.546202 .03188 1.522291 .621669 1.522291C1.195517 1.522291 1.362889 1.546202 1.41868 1.546202C1.45056 1.546202 1.570112 1.546202 1.570112 1.39477C1.570112 1.283188 1.458531 1.283188 1.362889 1.283188C.980324 1.283188 .980324 1.235367 .980324 1.163636C.980324 1.107846 1.123786 .541968 1.362889-.390535C1.466501-.207223 1.713574 .079701 2.14396 .079701C3.124284 .079701 4.144458-1.052055 4.144458-2.207721C4.144458-2.996762 3.634371-3.514819 2.996762-3.514819C2.518555-3.514819 2.13599-3.188045 1.904857-2.948941C1.737484-3.514819 1.203487-3.514819 1.123786-3.514819C.836862-3.514819 .637609-3.331507 .510087-3.084433C.326775-2.725778 .239103-2.319303 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.629639-2.83736 .773101-3.291656 1.099875-3.291656C1.299128-3.291656 1.354919-3.108344 1.354919-2.917061C1.354919-2.83736 1.323039-2.646077 1.307098-2.582316L.414446 .964384ZM1.880946-2.454795C1.920797-2.590286 1.920797-2.606227 2.040349-2.749689C2.343213-3.108344 2.685928-3.291656 2.972852-3.291656C3.371357-3.291656 3.52279-2.901121 3.52279-2.542466C3.52279-2.247572 3.347447-1.39477 3.108344-.924533C2.901121-.494147 2.518555-.143462 2.14396-.143462C1.601993-.143462 1.474471-.765131 1.474471-.820922C1.474471-.836862 1.490411-.924533 1.498381-.948443L1.880946-2.454795Z'/> +<path id='g4-115' d='M3.211955-2.996762C3.028643-2.964882 2.86127-2.82142 2.86127-2.622167C2.86127-2.478705 2.956912-2.375093 3.132254-2.375093C3.251806-2.375093 3.498879-2.462765 3.498879-2.82142C3.498879-3.315567 2.980822-3.514819 2.486675-3.514819C1.41868-3.514819 1.083935-2.757659 1.083935-2.351183C1.083935-2.271482 1.083935-1.984558 1.378829-1.761395C1.562142-1.617933 1.697634-1.594022 2.11208-1.514321C2.391034-1.458531 2.84533-1.378829 2.84533-.964384C2.84533-.757161 2.693898-.494147 2.470735-.342715C2.175841-.151432 1.785305-.143462 1.657783-.143462C1.466501-.143462 .924533-.175342 .72528-.494147C1.131756-.510087 1.187547-.836862 1.187547-.932503C1.187547-1.171606 .972354-1.227397 .876712-1.227397C.749191-1.227397 .422416-1.131756 .422416-.6934C.422416-.223163 .916563 .079701 1.657783 .079701C3.044583 .079701 3.339477-.900623 3.339477-1.235367C3.339477-1.952677 2.558406-2.10411 2.263512-2.1599C1.880946-2.231631 1.570112-2.287422 1.570112-2.622167C1.570112-2.765629 1.705604-3.291656 2.478705-3.291656C2.781569-3.291656 3.092403-3.203985 3.211955-2.996762Z'/> +<path id='g7-43' d='M3.474969-1.809215H5.818182C5.929763-1.809215 6.105106-1.809215 6.105106-1.992528S5.929763-2.175841 5.818182-2.175841H3.474969V-4.527024C3.474969-4.638605 3.474969-4.813948 3.291656-4.813948S3.108344-4.638605 3.108344-4.527024V-2.175841H.757161C.645579-2.175841 .470237-2.175841 .470237-1.992528S.645579-1.809215 .757161-1.809215H3.108344V.541968C3.108344 .653549 3.108344 .828892 3.291656 .828892S3.474969 .653549 3.474969 .541968V-1.809215Z'/> +<path id='g7-50' d='M2.247572-1.625903C2.375093-1.745455 2.709838-2.008468 2.83736-2.12005C3.331507-2.574346 3.801743-3.012702 3.801743-3.737983C3.801743-4.686426 3.004732-5.300125 2.008468-5.300125C1.052055-5.300125 .422416-4.574844 .422416-3.865504C.422416-3.474969 .73325-3.419178 .844832-3.419178C1.012204-3.419178 1.259278-3.53873 1.259278-3.841594C1.259278-4.25604 .860772-4.25604 .765131-4.25604C.996264-4.837858 1.530262-5.037111 1.920797-5.037111C2.662017-5.037111 3.044583-4.407472 3.044583-3.737983C3.044583-2.909091 2.462765-2.303362 1.522291-1.338979L.518057-.302864C.422416-.215193 .422416-.199253 .422416 0H3.57061L3.801743-1.42665H3.55467C3.53076-1.267248 3.466999-.868742 3.371357-.71731C3.323537-.653549 2.717808-.653549 2.590286-.653549H1.171606L2.247572-1.625903Z'/> +<path id='g7-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g7-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g5-110' d='M.96-.643636C.927272-.48 .861818-.229091 .861818-.174545C.861818 .021818 1.014545 .12 1.178181 .12C1.309091 .12 1.505454 .032727 1.581818-.185454C1.592727-.207273 1.723636-.72 1.78909-.992727L2.02909-1.974545C2.094545-2.214545 2.159999-2.454545 2.214545-2.705454C2.258181-2.890908 2.345454-3.207272 2.356363-3.250908C2.519999-3.58909 3.098181-4.581817 4.134544-4.581817C4.625453-4.581817 4.723635-4.178181 4.723635-3.818181C4.723635-3.141817 4.18909-1.745454 4.014544-1.276363C3.916363-1.025454 3.905453-.894545 3.905453-.774545C3.905453-.261818 4.287272 .12 4.799999 .12C5.825453 .12 6.229089-1.472727 6.229089-1.56C6.229089-1.66909 6.130907-1.66909 6.09818-1.66909C5.989089-1.66909 5.989089-1.636363 5.934544-1.472727C5.716362-.730909 5.356362-.12 4.821817-.12C4.636362-.12 4.559999-.229091 4.559999-.48C4.559999-.752727 4.658181-1.014545 4.756362-1.254545C4.963635-1.832727 5.421817-3.032726 5.421817-3.654544C5.421817-4.385453 4.952726-4.821817 4.167272-4.821817C3.185454-4.821817 2.650908-4.123635 2.465454-3.872726C2.410908-4.483635 1.963636-4.821817 1.461818-4.821817S.752727-4.396362 .643636-4.199999C.469091-3.82909 .316364-3.185454 .316364-3.141817C.316364-3.032726 .425454-3.032726 .447273-3.032726C.556363-3.032726 .567273-3.043636 .632727-3.283635C.818182-4.058181 1.036363-4.581817 1.429091-4.581817C1.647272-4.581817 1.767272-4.439999 1.767272-4.079999C1.767272-3.850908 1.734545-3.730908 1.592727-3.163635L.96-.643636Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -65.54)'> +<use x='70.734745' y='72.661996' xlink:href='#g5-110'/> +<use x='77.282761' y='66.911907' xlink:href='#g7-91'/> +<use x='79.635085' y='66.911907' xlink:href='#g4-108'/> +<use x='82.25721' y='66.911907' xlink:href='#g7-93'/> +<use x='77.282761' y='76.096079' xlink:href='#g4-87'/> +<use x='89.976903' y='72.661996' xlink:href='#g8-61'/> +<use x='101.492026' y='72.661996' xlink:href='#g2-98'/> +<use x='107.536036' y='66.506156' xlink:href='#g4-110'/> +<use x='112.674239' y='62.266501' xlink:href='#g6-91'/> +<use x='114.777438' y='62.266501' xlink:href='#g3-108'/> +<use x='117.178166' y='62.266501' xlink:href='#g0-0'/> +<use x='122.934319' y='62.266501' xlink:href='#g6-49'/> +<use x='126.587244' y='62.266501' xlink:href='#g6-93'/> +<use x='112.674239' y='69.280208' xlink:href='#g3-87'/> +<use x='129.188543' y='66.506156' xlink:href='#g7-43'/> +<use x='135.77505' y='66.506156' xlink:href='#g7-50'/> +<use x='140.009233' y='66.506156' xlink:href='#g4-112'/> +<use x='144.27201' y='63.69337' xlink:href='#g6-91'/> +<use x='146.375209' y='63.69337' xlink:href='#g3-108'/> +<use x='148.775937' y='63.69337' xlink:href='#g6-93'/> +<use x='151.377262' y='66.506156' xlink:href='#g1-0'/> +<use x='157.963769' y='66.506156' xlink:href='#g4-102'/> +<use x='162.910535' y='63.69337' xlink:href='#g6-91'/> +<use x='165.013734' y='63.69337' xlink:href='#g3-108'/> +<use x='167.414462' y='63.69337' xlink:href='#g6-93'/> +<rect x='107.536036' y='69.71656' height='.436352' width='62.479751'/> +<use x='133.265327' y='77.42473' xlink:href='#g4-115'/> +<use x='137.181245' y='75.072452' xlink:href='#g6-91'/> +<use x='139.284444' y='75.072452' xlink:href='#g3-108'/> +<use x='141.685172' y='75.072452' xlink:href='#g6-93'/> +<use x='173.635499' y='72.661996' xlink:href='#g8-43'/> +<use x='184.544573' y='72.661996' xlink:href='#g8-49'/> +<use x='189.999138' y='72.661996' xlink:href='#g2-99'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_padding_size.svg b/_resources/layer_padding_size.svg new file mode 100644 index 0000000..2aab54a --- /dev/null +++ b/_resources/layer_padding_size.svg @@ -0,0 +1,16 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='14.209887pt' height='14.953372pt' viewBox='-.237481 -.232383 14.209887 14.953372'> +<defs> +<path id='g0-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g2-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g2-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g1-112' d='M.514072 1.518306C.430386 1.876961 .382565 1.972603-.107597 1.972603C-.251059 1.972603-.37061 1.972603-.37061 2.199751C-.37061 2.223661-.358655 2.319303-.227148 2.319303C-.071731 2.319303 .095641 2.295392 .251059 2.295392H.765131C1.016189 2.295392 1.625903 2.319303 1.876961 2.319303C1.948692 2.319303 2.092154 2.319303 2.092154 2.10411C2.092154 1.972603 2.008468 1.972603 1.80523 1.972603C1.255293 1.972603 1.219427 1.888917 1.219427 1.793275C1.219427 1.649813 1.75741-.406476 1.829141-.681445C1.960648-.3467 2.283437 .119552 2.905106 .119552C4.25604 .119552 5.71457-1.637858 5.71457-3.395268C5.71457-4.495143 5.092902-5.272229 4.196264-5.272229C3.431133-5.272229 2.785554-4.531009 2.654047-4.363636C2.558406-4.961395 2.092154-5.272229 1.613948-5.272229C1.267248-5.272229 .992279-5.104857 .765131-4.65056C.549938-4.220174 .382565-3.490909 .382565-3.443088S.430386-3.335492 .514072-3.335492C.609714-3.335492 .621669-3.347447 .6934-3.622416C.872727-4.327771 1.099875-5.033126 1.578082-5.033126C1.853051-5.033126 1.948692-4.841843 1.948692-4.483188C1.948692-4.196264 1.912827-4.076712 1.865006-3.861519L.514072 1.518306ZM2.582316-3.730012C2.666002-4.064757 3.000747-4.411457 3.19203-4.578829C3.323537-4.698381 3.718057-5.033126 4.172354-5.033126C4.698381-5.033126 4.937484-4.507098 4.937484-3.88543C4.937484-3.311582 4.60274-1.960648 4.303861-1.338979C4.004981-.6934 3.455044-.119552 2.905106-.119552C2.092154-.119552 1.960648-1.147696 1.960648-1.195517C1.960648-1.231382 1.984558-1.327024 1.996513-1.3868L2.582316-3.730012Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.565683 -62.201193)'> +<use x='56.413267' y='65.753425' xlink:href='#g1-112'/> +<use x='62.28841' y='60.817239' xlink:href='#g2-91'/> +<use x='64.640734' y='60.817239' xlink:href='#g0-108'/> +<use x='67.262859' y='60.817239' xlink:href='#g2-93'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_stride_size.svg b/_resources/layer_stride_size.svg new file mode 100644 index 0000000..5dcf27b --- /dev/null +++ b/_resources/layer_stride_size.svg @@ -0,0 +1,16 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='13.383012pt' height='12.467654pt' viewBox='-.239051 -.232383 13.383012 12.467654'> +<defs> +<path id='g0-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g2-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g2-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g1-115' d='M2.725778-2.391034C2.929016-2.355168 3.251806-2.283437 3.323537-2.271482C3.478954-2.223661 4.016936-2.032379 4.016936-1.458531C4.016936-1.08792 3.682192-.119552 2.295392-.119552C2.044334-.119552 1.147696-.155417 .908593-.812951C1.3868-.753176 1.625903-1.123786 1.625903-1.3868C1.625903-1.637858 1.458531-1.769365 1.219427-1.769365C.956413-1.769365 .609714-1.566127 .609714-1.028144C.609714-.32279 1.327024 .119552 2.283437 .119552C4.100623 .119552 4.638605-1.219427 4.638605-1.841096C4.638605-2.020423 4.638605-2.355168 4.25604-2.737733C3.957161-3.024658 3.670237-3.084433 3.024658-3.21594C2.701868-3.287671 2.187796-3.395268 2.187796-3.93325C2.187796-4.172354 2.402989-5.033126 3.53873-5.033126C4.040847-5.033126 4.531009-4.841843 4.65056-4.411457C4.124533-4.411457 4.100623-3.957161 4.100623-3.945205C4.100623-3.694147 4.327771-3.622416 4.435367-3.622416C4.60274-3.622416 4.937484-3.753923 4.937484-4.25604S4.483188-5.272229 3.550685-5.272229C1.984558-5.272229 1.566127-4.040847 1.566127-3.550685C1.566127-2.642092 2.450809-2.450809 2.725778-2.391034Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.201193)'> +<use x='56.413267' y='65.753425' xlink:href='#g1-115'/> +<use x='61.927273' y='60.817239' xlink:href='#g2-91'/> +<use x='64.279597' y='60.817239' xlink:href='#g0-108'/> +<use x='66.901722' y='60.817239' xlink:href='#g2-93'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/layer_weights.svg b/_resources/layer_weights.svg new file mode 100644 index 0000000..e754f65 --- /dev/null +++ b/_resources/layer_weights.svg @@ -0,0 +1,40 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='127.157011pt' height='15.762355pt' viewBox='-.239051 -.232383 127.157011 15.762355'> +<defs> +<path id='g0-0' d='M5.571108-1.809215C5.69863-1.809215 5.873973-1.809215 5.873973-1.992528S5.69863-2.175841 5.571108-2.175841H1.004234C.876712-2.175841 .70137-2.175841 .70137-1.992528S.876712-1.809215 1.004234-1.809215H5.571108Z'/> +<path id='g1-2' d='M4.65056-3.323537L2.259527-5.702615C2.116065-5.846077 2.092154-5.869988 1.996513-5.869988C1.876961-5.869988 1.75741-5.762391 1.75741-5.630884C1.75741-5.547198 1.78132-5.523288 1.912827-5.391781L4.303861-2.988792L1.912827-.585803C1.78132-.454296 1.75741-.430386 1.75741-.3467C1.75741-.215193 1.876961-.107597 1.996513-.107597C2.092154-.107597 2.116065-.131507 2.259527-.274969L4.638605-2.654047L7.113325-.179328C7.137235-.167372 7.220922-.107597 7.292653-.107597C7.436115-.107597 7.531756-.215193 7.531756-.3467C7.531756-.37061 7.531756-.418431 7.49589-.478207C7.483935-.502117 5.583064-2.379078 4.985305-2.988792L7.173101-5.176588C7.232877-5.248319 7.412204-5.403736 7.47198-5.475467C7.483935-5.499377 7.531756-5.547198 7.531756-5.630884C7.531756-5.762391 7.436115-5.869988 7.292653-5.869988C7.197011-5.869988 7.149191-5.822167 7.017684-5.69066L4.65056-3.323537Z'/> +<path id='g2-99' d='M3.259776-3.052553C2.933001-3.012702 2.82939-2.765629 2.82939-2.606227C2.82939-2.375093 3.036613-2.311333 3.140224-2.311333C3.180075-2.311333 3.58655-2.343213 3.58655-2.82939S3.060523-3.514819 2.582316-3.514819C1.45056-3.514819 .350685-2.414944 .350685-1.299128C.350685-.541968 .868742 .079701 1.753425 .079701C3.012702 .079701 3.674222-.72528 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.769365-.143462C1.338979-.143462 .996264-.406476 .996264-1.012204C.996264-1.362889 1.155666-2.207721 1.530262-2.701868C1.880946-3.148194 2.279452-3.291656 2.590286-3.291656C2.685928-3.291656 3.052553-3.283686 3.259776-3.052553Z'/> +<path id='g2-108' d='M2.088169-5.292154C2.096139-5.308095 2.12005-5.411706 2.12005-5.419676C2.12005-5.459527 2.088169-5.531258 1.992528-5.531258L1.187547-5.467497C.892653-5.443587 .828892-5.435616 .828892-5.292154C.828892-5.180573 .940473-5.180573 1.036115-5.180573C1.41868-5.180573 1.41868-5.132752 1.41868-5.061021C1.41868-5.037111 1.41868-5.021171 1.378829-4.877709L.390535-.924533C.358655-.797011 .358655-.67746 .358655-.669489C.358655-.175342 .765131 .079701 1.163636 .079701C1.506351 .079701 1.689664-.191283 1.777335-.366625C1.920797-.629639 2.040349-1.099875 2.040349-1.139726C2.040349-1.187547 2.016438-1.243337 1.912827-1.243337C1.841096-1.243337 1.817186-1.203487 1.817186-1.195517C1.801245-1.171606 1.761395-1.028144 1.737484-.940473C1.617933-.478207 1.466501-.143462 1.179577-.143462C.988294-.143462 .932503-.326775 .932503-.518057C.932503-.669489 .956413-.757161 .980324-.860772L2.088169-5.292154Z'/> +<path id='g4-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g4-91' d='M2.1599 1.992528V1.625903H1.354919V-5.610959H2.1599V-5.977584H.988294V1.992528H2.1599Z'/> +<path id='g4-93' d='M1.354919-5.977584H.183313V-5.610959H.988294V1.625903H.183313V1.992528H1.354919V-5.977584Z'/> +<path id='g3-102' d='M5.332005-4.805978C5.571108-4.805978 5.66675-4.805978 5.66675-5.033126C5.66675-5.152677 5.571108-5.152677 5.355915-5.152677H4.387547C4.614695-6.38406 4.782067-7.232877 4.877709-7.615442C4.94944-7.902366 5.200498-8.177335 5.511333-8.177335C5.762391-8.177335 6.01345-8.069738 6.133001-7.962142C5.66675-7.914321 5.523288-7.567621 5.523288-7.364384C5.523288-7.12528 5.702615-6.981818 5.929763-6.981818C6.168867-6.981818 6.527522-7.185056 6.527522-7.639352C6.527522-8.141469 6.025405-8.416438 5.499377-8.416438C4.985305-8.416438 4.483188-8.033873 4.244085-7.567621C4.028892-7.149191 3.90934-6.718804 3.634371-5.152677H2.833375C2.606227-5.152677 2.486675-5.152677 2.486675-4.937484C2.486675-4.805978 2.558406-4.805978 2.797509-4.805978H3.56264C3.347447-3.694147 2.857285-.992279 2.582316 .286924C2.379078 1.327024 2.199751 2.199751 1.601993 2.199751C1.566127 2.199751 1.219427 2.199751 1.004234 1.972603C1.613948 1.924782 1.613948 1.398755 1.613948 1.3868C1.613948 1.147696 1.43462 1.004234 1.207472 1.004234C.968369 1.004234 .609714 1.207472 .609714 1.661768C.609714 2.175841 1.135741 2.438854 1.601993 2.438854C2.82142 2.438854 3.323537 .251059 3.455044-.3467C3.670237-1.267248 4.25604-4.447323 4.315816-4.805978H5.332005Z'/> +<path id='g3-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -62.201193)'> +<use x='56.413267' y='65.753425' xlink:href='#g3-102'/> +<use x='63.459704' y='60.817239' xlink:href='#g4-91'/> +<use x='65.812028' y='60.817239' xlink:href='#g2-108'/> +<use x='68.434153' y='60.817239' xlink:href='#g4-93'/> +<use x='73.941272' y='65.753425' xlink:href='#g1-2'/> +<use x='85.896432' y='65.753425' xlink:href='#g3-102'/> +<use x='92.942869' y='60.817239' xlink:href='#g4-91'/> +<use x='95.295193' y='60.817239' xlink:href='#g2-108'/> +<use x='97.917318' y='60.817239' xlink:href='#g4-93'/> +<use x='103.424437' y='65.753425' xlink:href='#g1-2'/> +<use x='115.379598' y='65.753425' xlink:href='#g3-110'/> +<use x='122.367203' y='60.817239' xlink:href='#g4-91'/> +<use x='124.719527' y='60.817239' xlink:href='#g2-108'/> +<use x='127.341652' y='60.817239' xlink:href='#g0-0'/> +<use x='133.928159' y='60.817239' xlink:href='#g4-49'/> +<use x='138.162342' y='60.817239' xlink:href='#g4-93'/> +<use x='122.367203' y='68.70894' xlink:href='#g2-99'/> +<use x='143.669461' y='65.753425' xlink:href='#g1-2'/> +<use x='155.624621' y='65.753425' xlink:href='#g3-110'/> +<use x='162.612227' y='60.817239' xlink:href='#g4-91'/> +<use x='164.964551' y='60.817239' xlink:href='#g2-108'/> +<use x='167.586676' y='60.817239' xlink:href='#g4-93'/> +<use x='162.612227' y='68.70894' xlink:href='#g2-99'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/leaky-relu.png b/_resources/leaky-relu.png new file mode 100644 index 0000000..7388aac Binary files /dev/null and b/_resources/leaky-relu.png differ diff --git a/_resources/lenet-5.png b/_resources/lenet-5.png new file mode 100644 index 0000000..7afe922 Binary files /dev/null and b/_resources/lenet-5.png differ diff --git a/_resources/lr-cost-function.png b/_resources/lr-cost-function.png new file mode 100644 index 0000000..1ffd1a5 Binary files /dev/null and b/_resources/lr-cost-function.png differ diff --git a/_resources/lr-eqn.svg b/_resources/lr-eqn.svg new file mode 100644 index 0000000..ba5e0af --- /dev/null +++ b/_resources/lr-eqn.svg @@ -0,0 +1,39 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='105.314876pt' height='14.309951pt' viewBox='-.299738 -.24919 105.314876 14.309951'> +<defs> +<path id='g0-84' d='M3.602491-4.821918C3.674222-5.108842 3.682192-5.124782 4.008966-5.124782H4.614695C5.443587-5.124782 5.539228-4.861768 5.539228-4.463263C5.539228-4.26401 5.491407-3.921295 5.483437-3.881445C5.467497-3.793773 5.459527-3.722042 5.459527-3.706102C5.459527-3.602491 5.531258-3.57858 5.579078-3.57858C5.66675-3.57858 5.69863-3.626401 5.72254-3.777833L5.937733-5.276214C5.937733-5.387796 5.842092-5.387796 5.69863-5.387796H1.004234C.804981-5.387796 .789041-5.387796 .73325-5.220423L.247073-3.841594C.231133-3.801743 .207223-3.737983 .207223-3.690162C.207223-3.626401 .263014-3.57858 .326775-3.57858C.414446-3.57858 .430386-3.618431 .478207-3.753923C.932503-5.029141 1.163636-5.124782 2.375093-5.124782H2.685928C2.925031-5.124782 2.933001-5.116812 2.933001-5.053051C2.933001-5.029141 2.901121-4.869738 2.893151-4.837858L1.841096-.653549C1.769365-.350685 1.745455-.263014 .916563-.263014C.661519-.263014 .581818-.263014 .581818-.111582C.581818-.103611 .581818 0 .71731 0C.932503 0 1.482441-.02391 1.697634-.02391H2.375093C2.598257-.02391 3.156164 0 3.379328 0C3.443088 0 3.56264 0 3.56264-.151432C3.56264-.263014 3.474969-.263014 3.259776-.263014C3.068493-.263014 3.004732-.263014 2.797509-.278954C2.542466-.302864 2.510585-.334745 2.510585-.438356C2.510585-.470237 2.518555-.502117 2.542466-.581818L3.602491-4.821918Z'/> +<path id='g0-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g2-40' d='M2.654047 1.992528C2.717808 1.992528 2.81345 1.992528 2.81345 1.896887C2.81345 1.865006 2.805479 1.857036 2.701868 1.753425C1.609963 .72528 1.338979-.757161 1.338979-1.992528C1.338979-4.28792 2.287422-5.363885 2.693898-5.730511C2.805479-5.834122 2.81345-5.842092 2.81345-5.881943S2.781569-5.977584 2.701868-5.977584C2.574346-5.977584 2.175841-5.571108 2.11208-5.499377C1.044085-4.383562 .820922-2.948941 .820922-1.992528C.820922-.207223 1.570112 1.227397 2.654047 1.992528Z'/> +<path id='g2-41' d='M2.462765-1.992528C2.462765-2.749689 2.335243-3.658281 1.841096-4.598755C1.45056-5.332005 .72528-5.977584 .581818-5.977584C.502117-5.977584 .478207-5.921793 .478207-5.881943C.478207-5.850062 .478207-5.834122 .573848-5.738481C1.689664-4.678456 1.944707-3.219925 1.944707-1.992528C1.944707 .294894 .996264 1.378829 .589788 1.745455C.486177 1.849066 .478207 1.857036 .478207 1.896887S.502117 1.992528 .581818 1.992528C.70934 1.992528 1.107846 1.586052 1.171606 1.514321C2.239601 .398506 2.462765-1.036115 2.462765-1.992528Z'/> +<path id='g1-27' d='M5.650908-4.06909C5.792726-4.06909 6.185453-4.06909 6.185453-4.439999C6.185453-4.701817 5.956362-4.701817 5.759998-4.701817H3.272726C1.625454-4.701817 .414545-2.901817 .414545-1.603636C.414545-.643636 1.058182 .12 2.050909 .12C3.338181 .12 4.78909-1.2 4.78909-2.879999C4.78909-3.065454 4.78909-3.58909 4.450908-4.06909H5.650908ZM2.061818-.12C1.527272-.12 1.090909-.512727 1.090909-1.298181C1.090909-1.625454 1.221818-2.519999 1.603636-3.163635C2.061818-3.916363 2.716363-4.06909 3.087272-4.06909C4.003635-4.06909 4.090908-3.34909 4.090908-3.010908C4.090908-2.498181 3.872726-1.603636 3.501817-1.047272C3.076363-.403636 2.487272-.12 2.061818-.12Z'/> +<path id='g1-98' d='M2.607272-7.450907C2.607272-7.461816 2.607272-7.570907 2.465454-7.570907C2.214545-7.570907 1.418181-7.483634 1.134545-7.461816C1.047272-7.450907 .927272-7.439998 .927272-7.243634C.927272-7.112725 1.025454-7.112725 1.189091-7.112725C1.712727-7.112725 1.734545-7.036362 1.734545-6.927271C1.734545-6.850907 1.636363-6.479998 1.581818-6.250907L.687273-2.694545C.556363-2.14909 .512727-1.974545 .512727-1.592727C.512727-.556363 1.090909 .12 1.898181 .12C3.185454 .12 4.527271-1.505454 4.527271-3.076363C4.527271-4.06909 3.94909-4.821817 3.076363-4.821817C2.574545-4.821817 2.127272-4.505453 1.799999-4.167272L2.607272-7.450907ZM1.581818-3.327272C1.647272-3.567272 1.647272-3.58909 1.745454-3.70909C2.279999-4.418181 2.770908-4.581817 3.054545-4.581817C3.447272-4.581817 3.741817-4.254544 3.741817-3.556363C3.741817-2.912726 3.381817-1.658181 3.185454-1.243636C2.825454-.512727 2.323636-.12 1.898181-.12C1.527272-.12 1.167272-.414545 1.167272-1.221818C1.167272-1.429091 1.167272-1.636363 1.341818-2.323636L1.581818-3.327272Z'/> +<path id='g1-119' d='M5.039999-3.698181C5.094544-3.938181 5.203635-4.341817 5.203635-4.407271C5.203635-4.603635 5.050908-4.701817 4.887271-4.701817C4.756362-4.701817 4.559999-4.614544 4.483635-4.396362C4.450908-4.319999 3.938181-2.236363 3.861817-1.952727C3.785453-1.625454 3.763635-1.429091 3.763635-1.232727C3.763635-1.112727 3.763635-1.090909 3.774544-1.036363C3.523635-.458182 3.196363-.12 2.770908-.12C1.898181-.12 1.898181-.927272 1.898181-1.112727C1.898181-1.461818 1.952727-1.887272 2.465454-3.22909C2.585454-3.556363 2.650908-3.70909 2.650908-3.927272C2.650908-4.418181 2.290908-4.821817 1.756363-4.821817C.72-4.821817 .316364-3.239999 .316364-3.141817C.316364-3.032726 .425454-3.032726 .447273-3.032726C.556363-3.032726 .567273-3.054545 .621818-3.22909C.916363-4.243635 1.341818-4.581817 1.723636-4.581817C1.821818-4.581817 1.996363-4.570908 1.996363-4.221817C1.996363-3.94909 1.876363-3.643635 1.799999-3.458181C1.32-2.170908 1.189091-1.66909 1.189091-1.254545C1.189091-.261818 1.919999 .12 2.738181 .12C2.923636 .12 3.436363 .12 3.872726-.643636C4.156362 .054545 4.90909 .12 5.236362 .12C6.054544 .12 6.534544-.567273 6.81818-1.221818C7.189089-2.072727 7.53818-3.534544 7.53818-4.058181C7.53818-4.658181 7.243634-4.821817 7.05818-4.821817C6.785453-4.821817 6.512725-4.538181 6.512725-4.298181C6.512725-4.156362 6.57818-4.090908 6.676362-4.003635C6.796362-3.883635 7.069089-3.599999 7.069089-3.076363C7.069089-2.705454 6.752725-1.636363 6.469089-1.08C6.185453-.501818 5.803635-.12 5.269089-.12C4.756362-.12 4.461817-.447273 4.461817-1.069091C4.461817-1.374545 4.538181-1.712727 4.581817-1.865454L5.039999-3.698181Z'/> +<path id='g1-120' d='M3.643635-3.294545C3.70909-3.578181 3.959999-4.581817 4.723635-4.581817C4.77818-4.581817 5.039999-4.581817 5.269089-4.439999C4.963635-4.385453 4.745453-4.112726 4.745453-3.850908C4.745453-3.676363 4.865453-3.46909 5.159999-3.46909C5.399998-3.46909 5.749089-3.665454 5.749089-4.101817C5.749089-4.66909 5.105453-4.821817 4.734544-4.821817C4.101817-4.821817 3.719999-4.243635 3.58909-3.992726C3.316363-4.712726 2.727272-4.821817 2.410908-4.821817C1.276363-4.821817 .654545-3.414545 .654545-3.141817C.654545-3.032726 .763636-3.032726 .785454-3.032726C.872727-3.032726 .905454-3.054545 .927272-3.152726C1.298181-4.30909 2.018181-4.581817 2.38909-4.581817C2.596363-4.581817 2.978181-4.483635 2.978181-3.850908C2.978181-3.512726 2.792726-2.781817 2.38909-1.254545C2.214545-.578182 1.832727-.12 1.352727-.12C1.287272-.12 1.036363-.12 .807273-.261818C1.08-.316364 1.32-.545454 1.32-.850909C1.32-1.145454 1.08-1.232727 .916363-1.232727C.589091-1.232727 .316364-.949091 .316364-.6C.316364-.098182 .861818 .12 1.341818 .12C2.061818 .12 2.454545-.643636 2.487272-.709091C2.618181-.305454 3.010908 .12 3.665454 .12C4.78909 .12 5.410908-1.287272 5.410908-1.56C5.410908-1.66909 5.312726-1.66909 5.279999-1.66909C5.181817-1.66909 5.159999-1.625454 5.13818-1.54909C4.77818-.381818 4.036363-.12 3.687272-.12C3.261817-.12 3.087272-.469091 3.087272-.84C3.087272-1.08 3.152726-1.32 3.272726-1.799999L3.643635-3.294545Z'/> +<path id='g1-121' d='M5.301817-4.156362C5.345453-4.30909 5.345453-4.330908 5.345453-4.407271C5.345453-4.603635 5.192726-4.701817 5.02909-4.701817C4.919999-4.701817 4.745453-4.636362 4.647271-4.472726C4.625453-4.418181 4.538181-4.079999 4.494544-3.883635C4.418181-3.599999 4.341817-3.305454 4.276362-3.010908L3.785453-1.047272C3.741817-.883636 3.272726-.12 2.552727-.12C1.996363-.12 1.876363-.6 1.876363-1.003636C1.876363-1.505454 2.061818-2.181818 2.432727-3.141817C2.607272-3.58909 2.650908-3.70909 2.650908-3.927272C2.650908-4.418181 2.301818-4.821817 1.756363-4.821817C.72-4.821817 .316364-3.239999 .316364-3.141817C.316364-3.032726 .425454-3.032726 .447273-3.032726C.556363-3.032726 .567273-3.054545 .621818-3.22909C.916363-4.254544 1.352727-4.581817 1.723636-4.581817C1.810909-4.581817 1.996363-4.581817 1.996363-4.232726C1.996363-3.959999 1.887272-3.676363 1.810909-3.46909C1.374545-2.312727 1.178181-1.690909 1.178181-1.178181C1.178181-.207273 1.865454 .12 2.50909 .12C2.934545 .12 3.305454-.065455 3.610908-.370909C3.46909 .196364 3.338181 .730909 2.901817 1.309091C2.618181 1.68 2.203636 1.996363 1.701818 1.996363C1.54909 1.996363 1.058182 1.963636 .872727 1.538181C1.047272 1.538181 1.189091 1.538181 1.341818 1.407272C1.450909 1.309091 1.56 1.167272 1.56 .96C1.56 .621818 1.265454 .578182 1.156363 .578182C.905454 .578182 .545454 .752727 .545454 1.287272C.545454 1.832727 1.025454 2.236363 1.701818 2.236363C2.825454 2.236363 3.94909 1.243636 4.254544 .010909L5.301817-4.156362Z'/> +<path id='g3-40' d='M3.610908 2.618181C3.610908 2.585454 3.610908 2.563636 3.425454 2.378181C2.061818 1.003636 1.712727-1.058182 1.712727-2.727272C1.712727-4.625453 2.127272-6.523635 3.46909-7.887271C3.610908-8.01818 3.610908-8.039998 3.610908-8.072725C3.610908-8.149089 3.567272-8.181816 3.501817-8.181816C3.392726-8.181816 2.410908-7.439998 1.767272-6.054544C1.210909-4.854544 1.08-3.643635 1.08-2.727272C1.08-1.876363 1.2-.556363 1.799999 .676363C2.454545 2.018181 3.392726 2.727272 3.501817 2.727272C3.567272 2.727272 3.610908 2.694545 3.610908 2.618181Z'/> +<path id='g3-41' d='M3.152726-2.727272C3.152726-3.578181 3.032726-4.89818 2.432727-6.130907C1.778181-7.472725 .84-8.181816 .730909-8.181816C.665454-8.181816 .621818-8.13818 .621818-8.072725C.621818-8.039998 .621818-8.01818 .829091-7.821816C1.898181-6.741816 2.519999-5.007271 2.519999-2.727272C2.519999-.861818 2.116363 1.058182 .763636 2.432727C.621818 2.563636 .621818 2.585454 .621818 2.618181C.621818 2.683636 .665454 2.727272 .730909 2.727272C.84 2.727272 1.821818 1.985454 2.465454 .6C3.021817-.6 3.152726-1.810909 3.152726-2.727272Z'/> +<path id='g3-43' d='M4.461817-2.50909H7.505452C7.65818-2.50909 7.865452-2.50909 7.865452-2.727272S7.65818-2.945454 7.505452-2.945454H4.461817V-5.999998C4.461817-6.152726 4.461817-6.359998 4.243635-6.359998S4.025453-6.152726 4.025453-5.999998V-2.945454H.970909C.818182-2.945454 .610909-2.945454 .610909-2.727272S.818182-2.50909 .970909-2.50909H4.025453V.545454C4.025453 .698182 4.025453 .905454 4.243635 .905454S4.461817 .698182 4.461817 .545454V-2.50909Z'/> +<path id='g3-61' d='M7.494543-3.567272C7.65818-3.567272 7.865452-3.567272 7.865452-3.785453S7.65818-4.003635 7.505452-4.003635H.970909C.818182-4.003635 .610909-4.003635 .610909-3.785453S.818182-3.567272 .981818-3.567272H7.494543ZM7.505452-1.450909C7.65818-1.450909 7.865452-1.450909 7.865452-1.66909S7.65818-1.887272 7.494543-1.887272H.981818C.818182-1.887272 .610909-1.887272 .610909-1.66909S.818182-1.450909 .970909-1.450909H7.505452Z'/> +<path id='g3-94' d='M2.727272-7.570907L1.265454-6.087271L1.461818-5.890907L2.727272-7.003634L3.981817-5.890907L4.178181-6.087271L2.727272-7.570907Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -66.69976)'> +<use x='71.483493' y='68.742217' xlink:href='#g3-94'/> +<use x='70.734745' y='68.742217' xlink:href='#g1-121'/> +<use x='76.474675' y='64.783399' xlink:href='#g2-40'/> +<use x='79.767928' y='64.783399' xlink:href='#g0-105'/> +<use x='82.651068' y='64.783399' xlink:href='#g2-41'/> +<use x='89.47269' y='68.742217' xlink:href='#g3-61'/> +<use x='100.987813' y='68.742217' xlink:href='#g1-27'/> +<use x='107.612819' y='68.742217' xlink:href='#g3-40'/> +<use x='111.855257' y='68.742217' xlink:href='#g1-119'/> +<use x='119.958811' y='64.783399' xlink:href='#g0-84'/> +<use x='126.563302' y='68.742217' xlink:href='#g1-120'/> +<use x='132.798156' y='64.783399' xlink:href='#g2-40'/> +<use x='136.091409' y='64.783399' xlink:href='#g0-105'/> +<use x='138.974549' y='64.783399' xlink:href='#g2-41'/> +<use x='145.190132' y='68.742217' xlink:href='#g3-43'/> +<use x='156.099206' y='68.742217' xlink:href='#g1-98'/> +<use x='160.781024' y='68.742217' xlink:href='#g3-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/lr-input.svg b/_resources/lr-input.svg new file mode 100644 index 0000000..4ca5458 --- /dev/null +++ b/_resources/lr-input.svg @@ -0,0 +1,49 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='148.374438pt' height='14.309951pt' viewBox='-.299738 -.24919 148.374438 14.309951'> +<defs> +<path id='g1-109' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.745455-1.928767 1.793275-2.12005 1.809215-2.199751C1.825156-2.239601 2.088169-2.757659 2.438854-3.020672C2.709838-3.227895 2.972852-3.291656 3.196015-3.291656C3.490909-3.291656 3.650311-3.116314 3.650311-2.749689C3.650311-2.558406 3.602491-2.375093 3.514819-2.016438C3.459029-1.809215 3.323537-1.275218 3.275716-1.060025L3.156164-.581818C3.116314-.446326 3.060523-.207223 3.060523-.167372C3.060523 .01594 3.211955 .079701 3.315567 .079701C3.459029 .079701 3.57858-.01594 3.634371-.111582C3.658281-.159402 3.722042-.430386 3.761893-.597758L3.945205-1.307098C3.969116-1.42665 4.048817-1.729514 4.072727-1.849066C4.184309-2.279452 4.184309-2.287422 4.367621-2.550436C4.630635-2.940971 5.00523-3.291656 5.539228-3.291656C5.826152-3.291656 5.993524-3.124284 5.993524-2.749689C5.993524-2.311333 5.65878-1.39477 5.507347-1.012204C5.427646-.804981 5.403736-.749191 5.403736-.597758C5.403736-.143462 5.778331 .079701 6.121046 .079701C6.902117 .079701 7.228892-1.036115 7.228892-1.139726C7.228892-1.219427 7.165131-1.243337 7.10934-1.243337C7.013699-1.243337 6.997758-1.187547 6.973848-1.107846C6.782565-.446326 6.447821-.143462 6.144956-.143462C6.017435-.143462 5.953674-.223163 5.953674-.406476S6.017435-.765131 6.097136-.964384C6.216687-1.267248 6.567372-2.183811 6.567372-2.630137C6.567372-3.227895 6.152927-3.514819 5.579078-3.514819C5.029141-3.514819 4.574844-3.227895 4.216189-2.733748C4.152428-3.371357 3.642341-3.514819 3.227895-3.514819C2.86127-3.514819 2.375093-3.387298 1.936737-2.81345C1.880946-3.291656 1.498381-3.514819 1.123786-3.514819C.844832-3.514819 .645579-3.347447 .510087-3.076463C.318804-2.701868 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g3-40' d='M2.654047 1.992528C2.717808 1.992528 2.81345 1.992528 2.81345 1.896887C2.81345 1.865006 2.805479 1.857036 2.701868 1.753425C1.609963 .72528 1.338979-.757161 1.338979-1.992528C1.338979-4.28792 2.287422-5.363885 2.693898-5.730511C2.805479-5.834122 2.81345-5.842092 2.81345-5.881943S2.781569-5.977584 2.701868-5.977584C2.574346-5.977584 2.175841-5.571108 2.11208-5.499377C1.044085-4.383562 .820922-2.948941 .820922-1.992528C.820922-.207223 1.570112 1.227397 2.654047 1.992528Z'/> +<path id='g3-41' d='M2.462765-1.992528C2.462765-2.749689 2.335243-3.658281 1.841096-4.598755C1.45056-5.332005 .72528-5.977584 .581818-5.977584C.502117-5.977584 .478207-5.921793 .478207-5.881943C.478207-5.850062 .478207-5.834122 .573848-5.738481C1.689664-4.678456 1.944707-3.219925 1.944707-1.992528C1.944707 .294894 .996264 1.378829 .589788 1.745455C.486177 1.849066 .478207 1.857036 .478207 1.896887S.502117 1.992528 .581818 1.992528C.70934 1.992528 1.107846 1.586052 1.171606 1.514321C2.239601 .398506 2.462765-1.036115 2.462765-1.992528Z'/> +<path id='g3-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g2-58' d='M2.094545-.578182C2.094545-.894545 1.832727-1.156363 1.516363-1.156363S.938182-.894545 .938182-.578182S1.2 0 1.516363 0S2.094545-.261818 2.094545-.578182Z'/> +<path id='g2-59' d='M2.214545-.010909C2.214545-.730909 1.941818-1.156363 1.516363-1.156363C1.156363-1.156363 .938182-.883636 .938182-.578182C.938182-.283636 1.156363 0 1.516363 0C1.647272 0 1.78909-.043636 1.898181-.141818C1.930909-.163636 1.941818-.174545 1.952727-.174545S1.974545-.163636 1.974545-.010909C1.974545 .796363 1.592727 1.450909 1.232727 1.810909C1.112727 1.930909 1.112727 1.952727 1.112727 1.985454C1.112727 2.061818 1.167272 2.105454 1.221818 2.105454C1.341818 2.105454 2.214545 1.265454 2.214545-.010909Z'/> +<path id='g2-120' d='M3.643635-3.294545C3.70909-3.578181 3.959999-4.581817 4.723635-4.581817C4.77818-4.581817 5.039999-4.581817 5.269089-4.439999C4.963635-4.385453 4.745453-4.112726 4.745453-3.850908C4.745453-3.676363 4.865453-3.46909 5.159999-3.46909C5.399998-3.46909 5.749089-3.665454 5.749089-4.101817C5.749089-4.66909 5.105453-4.821817 4.734544-4.821817C4.101817-4.821817 3.719999-4.243635 3.58909-3.992726C3.316363-4.712726 2.727272-4.821817 2.410908-4.821817C1.276363-4.821817 .654545-3.414545 .654545-3.141817C.654545-3.032726 .763636-3.032726 .785454-3.032726C.872727-3.032726 .905454-3.054545 .927272-3.152726C1.298181-4.30909 2.018181-4.581817 2.38909-4.581817C2.596363-4.581817 2.978181-4.483635 2.978181-3.850908C2.978181-3.512726 2.792726-2.781817 2.38909-1.254545C2.214545-.578182 1.832727-.12 1.352727-.12C1.287272-.12 1.036363-.12 .807273-.261818C1.08-.316364 1.32-.545454 1.32-.850909C1.32-1.145454 1.08-1.232727 .916363-1.232727C.589091-1.232727 .316364-.949091 .316364-.6C.316364-.098182 .861818 .12 1.341818 .12C2.061818 .12 2.454545-.643636 2.487272-.709091C2.618181-.305454 3.010908 .12 3.665454 .12C4.78909 .12 5.410908-1.287272 5.410908-1.56C5.410908-1.66909 5.312726-1.66909 5.279999-1.66909C5.181817-1.66909 5.159999-1.625454 5.13818-1.54909C4.77818-.381818 4.036363-.12 3.687272-.12C3.261817-.12 3.087272-.469091 3.087272-.84C3.087272-1.08 3.152726-1.32 3.272726-1.799999L3.643635-3.294545Z'/> +<path id='g2-121' d='M5.301817-4.156362C5.345453-4.30909 5.345453-4.330908 5.345453-4.407271C5.345453-4.603635 5.192726-4.701817 5.02909-4.701817C4.919999-4.701817 4.745453-4.636362 4.647271-4.472726C4.625453-4.418181 4.538181-4.079999 4.494544-3.883635C4.418181-3.599999 4.341817-3.305454 4.276362-3.010908L3.785453-1.047272C3.741817-.883636 3.272726-.12 2.552727-.12C1.996363-.12 1.876363-.6 1.876363-1.003636C1.876363-1.505454 2.061818-2.181818 2.432727-3.141817C2.607272-3.58909 2.650908-3.70909 2.650908-3.927272C2.650908-4.418181 2.301818-4.821817 1.756363-4.821817C.72-4.821817 .316364-3.239999 .316364-3.141817C.316364-3.032726 .425454-3.032726 .447273-3.032726C.556363-3.032726 .567273-3.054545 .621818-3.22909C.916363-4.254544 1.352727-4.581817 1.723636-4.581817C1.810909-4.581817 1.996363-4.581817 1.996363-4.232726C1.996363-3.959999 1.887272-3.676363 1.810909-3.46909C1.374545-2.312727 1.178181-1.690909 1.178181-1.178181C1.178181-.207273 1.865454 .12 2.50909 .12C2.934545 .12 3.305454-.065455 3.610908-.370909C3.46909 .196364 3.338181 .730909 2.901817 1.309091C2.618181 1.68 2.203636 1.996363 1.701818 1.996363C1.54909 1.996363 1.058182 1.963636 .872727 1.538181C1.047272 1.538181 1.189091 1.538181 1.341818 1.407272C1.450909 1.309091 1.56 1.167272 1.56 .96C1.56 .621818 1.265454 .578182 1.156363 .578182C.905454 .578182 .545454 .752727 .545454 1.287272C.545454 1.832727 1.025454 2.236363 1.701818 2.236363C2.825454 2.236363 3.94909 1.243636 4.254544 .010909L5.301817-4.156362Z'/> +<path id='g4-40' d='M3.610908 2.618181C3.610908 2.585454 3.610908 2.563636 3.425454 2.378181C2.061818 1.003636 1.712727-1.058182 1.712727-2.727272C1.712727-4.625453 2.127272-6.523635 3.46909-7.887271C3.610908-8.01818 3.610908-8.039998 3.610908-8.072725C3.610908-8.149089 3.567272-8.181816 3.501817-8.181816C3.392726-8.181816 2.410908-7.439998 1.767272-6.054544C1.210909-4.854544 1.08-3.643635 1.08-2.727272C1.08-1.876363 1.2-.556363 1.799999 .676363C2.454545 2.018181 3.392726 2.727272 3.501817 2.727272C3.567272 2.727272 3.610908 2.694545 3.610908 2.618181Z'/> +<path id='g4-41' d='M3.152726-2.727272C3.152726-3.578181 3.032726-4.89818 2.432727-6.130907C1.778181-7.472725 .84-8.181816 .730909-8.181816C.665454-8.181816 .621818-8.13818 .621818-8.072725C.621818-8.039998 .621818-8.01818 .829091-7.821816C1.898181-6.741816 2.519999-5.007271 2.519999-2.727272C2.519999-.861818 2.116363 1.058182 .763636 2.432727C.621818 2.563636 .621818 2.585454 .621818 2.618181C.621818 2.683636 .665454 2.727272 .730909 2.727272C.84 2.727272 1.821818 1.985454 2.465454 .6C3.021817-.6 3.152726-1.810909 3.152726-2.727272Z'/> +<path id='g0-102' d='M3.087272-6.730907C3.087272-7.167271 3.370908-7.865452 4.559999-7.941816C4.614544-7.952725 4.658181-7.996361 4.658181-8.061816C4.658181-8.181816 4.570908-8.181816 4.450908-8.181816C3.359999-8.181816 2.367272-7.625452 2.356363-6.81818V-4.330908C2.356363-3.905453 2.356363-3.556363 1.919999-3.196363C1.538181-2.879999 1.123636-2.858181 .883636-2.847272C.829091-2.836363 .785454-2.792726 .785454-2.727272C.785454-2.618181 .850909-2.618181 .96-2.607272C1.68-2.563636 2.203636-2.170908 2.323636-1.636363C2.356363-1.516363 2.356363-1.494545 2.356363-1.101818V1.058182C2.356363 1.516363 2.356363 1.865454 2.879999 2.279999C3.305454 2.607272 4.025453 2.727272 4.450908 2.727272C4.570908 2.727272 4.658181 2.727272 4.658181 2.607272C4.658181 2.498181 4.592726 2.498181 4.483635 2.487272C3.796363 2.443636 3.261817 2.094545 3.119999 1.538181C3.087272 1.44 3.087272 1.418181 3.087272 1.025454V-1.265454C3.087272-1.767272 2.999999-1.952727 2.650908-2.301818C2.421818-2.530908 2.105454-2.639999 1.799999-2.727272C2.694545-2.978181 3.087272-3.479999 3.087272-4.112726V-6.730907Z'/> +<path id='g0-103' d='M2.356363 1.276363C2.356363 1.712727 2.072727 2.410908 .883636 2.487272C.829091 2.498181 .785454 2.541817 .785454 2.607272C.785454 2.727272 .905454 2.727272 1.003636 2.727272C2.061818 2.727272 3.076363 2.192727 3.087272 1.363636V-1.123636C3.087272-1.54909 3.087272-1.898181 3.523635-2.258181C3.905453-2.574545 4.319999-2.596363 4.559999-2.607272C4.614544-2.618181 4.658181-2.661817 4.658181-2.727272C4.658181-2.836363 4.592726-2.836363 4.483635-2.847272C3.763635-2.890908 3.239999-3.283635 3.119999-3.818181C3.087272-3.938181 3.087272-3.959999 3.087272-4.352726V-6.512725C3.087272-6.970907 3.087272-7.319998 2.563636-7.734543C2.127272-8.072725 1.374545-8.181816 1.003636-8.181816C.905454-8.181816 .785454-8.181816 .785454-8.061816C.785454-7.952725 .850909-7.952725 .96-7.941816C1.647272-7.89818 2.181818-7.549089 2.323636-6.992725C2.356363-6.894544 2.356363-6.872725 2.356363-6.479998V-4.18909C2.356363-3.687272 2.443636-3.501817 2.792726-3.152726C3.021817-2.923636 3.338181-2.814545 3.643635-2.727272C2.74909-2.476363 2.356363-1.974545 2.356363-1.341818V1.276363Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -66.69976)'> +<use x='70.734745' y='68.742217' xlink:href='#g0-102'/> +<use x='76.189309' y='68.742217' xlink:href='#g4-40'/> +<use x='80.431748' y='68.742217' xlink:href='#g2-120'/> +<use x='86.666601' y='64.783399' xlink:href='#g3-40'/> +<use x='89.959855' y='64.783399' xlink:href='#g3-49'/> +<use x='94.194038' y='64.783399' xlink:href='#g3-41'/> +<use x='97.985423' y='68.742217' xlink:href='#g2-59'/> +<use x='102.833883' y='68.742217' xlink:href='#g2-121'/> +<use x='108.573813' y='64.783399' xlink:href='#g3-40'/> +<use x='111.867066' y='64.783399' xlink:href='#g3-49'/> +<use x='116.101249' y='64.783399' xlink:href='#g3-41'/> +<use x='119.892635' y='68.742217' xlink:href='#g4-41'/> +<use x='124.135073' y='68.742217' xlink:href='#g2-59'/> +<use x='128.983533' y='68.742217' xlink:href='#g2-58'/> +<use x='132.013844' y='68.742217' xlink:href='#g2-58'/> +<use x='135.044155' y='68.742217' xlink:href='#g2-58'/> +<use x='138.074467' y='68.742217' xlink:href='#g2-59'/> +<use x='142.922927' y='68.742217' xlink:href='#g4-40'/> +<use x='147.165365' y='68.742217' xlink:href='#g2-120'/> +<use x='153.400219' y='64.783399' xlink:href='#g3-40'/> +<use x='156.693472' y='64.783399' xlink:href='#g1-109'/> +<use x='164.183998' y='64.783399' xlink:href='#g3-41'/> +<use x='167.975384' y='68.742217' xlink:href='#g2-59'/> +<use x='172.823843' y='68.742217' xlink:href='#g2-121'/> +<use x='178.563774' y='64.783399' xlink:href='#g3-40'/> +<use x='181.857027' y='64.783399' xlink:href='#g1-109'/> +<use x='189.347553' y='64.783399' xlink:href='#g3-41'/> +<use x='193.138939' y='68.742217' xlink:href='#g4-41'/> +<use x='197.381377' y='68.742217' xlink:href='#g0-103'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/lr-loss-function.png b/_resources/lr-loss-function.png new file mode 100644 index 0000000..ff8ee63 Binary files /dev/null and b/_resources/lr-loss-function.png differ diff --git a/_resources/lr-sigmoid.svg b/_resources/lr-sigmoid.svg new file mode 100644 index 0000000..7644414 --- /dev/null +++ b/_resources/lr-sigmoid.svg @@ -0,0 +1,42 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='90.502887pt' height='19.868588pt' viewBox='-.299738 -.24919 90.502887 19.868588'> +<defs> +<path id='g4-40' d='M2.379078 1.350934C1.721544 .848817 1.183562-.095641 1.183562-1.494396C1.183562-2.875218 1.709589-3.831631 2.379078-4.339726C2.379078-4.345704 2.397011-4.369614 2.397011-4.399502C2.397011-4.435367 2.373101-4.483188 2.30137-4.483188C2.193773-4.483188 .753176-3.544707 .753176-1.494396C.753176 .549938 2.187796 1.494396 2.30137 1.494396C2.373101 1.494396 2.397011 1.446575 2.397011 1.41071C2.397011 1.380822 2.379078 1.356912 2.379078 1.350934Z'/> +<path id='g4-41' d='M2.116065-1.494396C2.116065-3.53873 .681445-4.483188 .56787-4.483188C.496139-4.483188 .478207-4.42939 .478207-4.399502C.478207-4.363636 .484184-4.357659 .591781-4.262017C1.129763-3.801743 1.685679-2.917061 1.685679-1.494396C1.685679-.227148 1.243337 .723288 .56787 1.291158C.484184 1.368867 .478207 1.374844 .478207 1.41071C.478207 1.440598 .496139 1.494396 .56787 1.494396C.675467 1.494396 2.116065 .555915 2.116065-1.494396Z'/> +<path id='g1-105' d='M2.080199-3.730012C2.080199-3.873474 1.972603-3.969116 1.835118-3.969116C1.673724-3.969116 1.500374-3.813699 1.500374-3.640349C1.500374-3.490909 1.60797-3.401245 1.739477-3.401245C1.93076-3.401245 2.080199-3.580573 2.080199-3.730012ZM1.721544-1.643836C1.745455-1.703611 1.799253-1.847073 1.823163-1.900872C1.841096-1.95467 1.865006-2.014446 1.865006-2.116065C1.865006-2.450809 1.566127-2.636115 1.267248-2.636115C.657534-2.636115 .364633-1.847073 .364633-1.715567C.364633-1.685679 .388543-1.63188 .472229-1.63188S.573848-1.667746 .591781-1.721544C.759153-2.30137 1.075965-2.438854 1.243337-2.438854C1.362889-2.438854 1.404732-2.361146 1.404732-2.223661C1.404732-2.10411 1.368867-2.014446 1.356912-1.972603L1.046077-1.207472C.974346-1.034122 .974346-1.022167 .896638-.818929C.818929-.639601 .789041-.561893 .789041-.460274C.789041-.155417 1.06401 .059776 1.392777 .059776C1.996513 .059776 2.295392-.729265 2.295392-.860772C2.295392-.872727 2.289415-.944458 2.181818-.944458C2.098132-.944458 2.092154-.91457 2.056289-.800996C1.960648-.496139 1.715567-.137484 1.41071-.137484C1.303113-.137484 1.249315-.209215 1.249315-.352677C1.249315-.472229 1.285181-.561893 1.362889-.747198L1.721544-1.643836Z'/> +<path id='g1-122' d='M1.129763-.537983C1.315068-.71731 1.524284-.89066 1.841096-1.129763C2.247572-1.440598 2.486675-1.625903 2.743711-1.865006C3.084433-2.175841 3.311582-2.492653 3.311582-2.552428C3.311582-2.636115 3.227895-2.636115 3.203985-2.636115C3.144209-2.636115 3.126276-2.624159 3.084433-2.558406C2.976837-2.402989 2.815442-2.223661 2.624159-2.223661C2.480697-2.223661 2.355168-2.32528 2.27746-2.391034C2.122042-2.52254 1.984558-2.636115 1.763387-2.636115C1.26127-2.636115 .926526-2.110087 .926526-1.93076C.926526-1.865006 .974346-1.841096 1.034122-1.841096C1.111831-1.841096 1.129763-1.876961 1.147696-1.924782C1.231382-2.133998 1.566127-2.15193 1.691656-2.15193C1.847073-2.15193 2.002491-2.122042 2.122042-2.092154C2.426899-2.032379 2.492653-2.032379 2.624159-2.026401C2.486675-1.894894 2.343213-1.763387 1.95467-1.470486C1.470486-1.099875 1.291158-.962391 1.0401-.729265C.735243-.454296 .448319-.101619 .448319-.02391C.448319 .059776 .537983 .059776 .561893 .059776C.621669 .059776 .633624 .047821 .669489 0C.908593-.316812 1.141719-.352677 1.23736-.352677C1.380822-.352677 1.482441-.274969 1.590037-.185305C1.769365-.047821 1.894894 .059776 2.110087 .059776C2.773599 .059776 3.198007-.669489 3.198007-.884682C3.198007-.938481 3.144209-.968369 3.084433-.968369C3.000747-.968369 2.988792-.932503 2.970859-.872727C2.851308-.54396 2.456787-.424408 2.175841-.424408C2.032379-.424408 1.888917-.454296 1.733499-.484184C1.554172-.52005 1.41071-.549938 1.267248-.549938C1.23736-.549938 1.183562-.549938 1.129763-.537983Z'/> +<path id='g0-0' d='M4.758157-1.338979C4.853798-1.338979 5.003238-1.338979 5.003238-1.494396S4.853798-1.649813 4.758157-1.649813H.992279C.896638-1.649813 .747198-1.649813 .747198-1.494396S.896638-1.338979 .992279-1.338979H4.758157Z'/> +<path id='g2-101' d='M1.601993-1.817186C1.777335-1.817186 2.375093-1.825156 2.797509-1.976588C3.490909-2.215691 3.514819-2.693898 3.514819-2.81345C3.514819-3.259776 3.092403-3.514819 2.574346-3.514819C1.673724-3.514819 .390535-2.805479 .390535-1.39477C.390535-.581818 .884682 .079701 1.761395 .079701C3.004732 .079701 3.674222-.71731 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.777335-.143462C1.195517-.143462 1.020174-.637609 1.020174-1.083935C1.020174-1.323039 1.091905-1.681694 1.123786-1.817186H1.601993ZM1.187547-2.040349C1.44259-3.012702 2.16787-3.291656 2.574346-3.291656C2.893151-3.291656 3.196015-3.132254 3.196015-2.81345C3.196015-2.040349 1.888917-2.040349 1.554172-2.040349H1.187547Z'/> +<path id='g2-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g5-40' d='M2.654047 1.992528C2.717808 1.992528 2.81345 1.992528 2.81345 1.896887C2.81345 1.865006 2.805479 1.857036 2.701868 1.753425C1.609963 .72528 1.338979-.757161 1.338979-1.992528C1.338979-4.28792 2.287422-5.363885 2.693898-5.730511C2.805479-5.834122 2.81345-5.842092 2.81345-5.881943S2.781569-5.977584 2.701868-5.977584C2.574346-5.977584 2.175841-5.571108 2.11208-5.499377C1.044085-4.383562 .820922-2.948941 .820922-1.992528C.820922-.207223 1.570112 1.227397 2.654047 1.992528Z'/> +<path id='g5-41' d='M2.462765-1.992528C2.462765-2.749689 2.335243-3.658281 1.841096-4.598755C1.45056-5.332005 .72528-5.977584 .581818-5.977584C.502117-5.977584 .478207-5.921793 .478207-5.881943C.478207-5.850062 .478207-5.834122 .573848-5.738481C1.689664-4.678456 1.944707-3.219925 1.944707-1.992528C1.944707 .294894 .996264 1.378829 .589788 1.745455C.486177 1.849066 .478207 1.857036 .478207 1.896887S.502117 1.992528 .581818 1.992528C.70934 1.992528 1.107846 1.586052 1.171606 1.514321C2.239601 .398506 2.462765-1.036115 2.462765-1.992528Z'/> +<path id='g5-43' d='M3.474969-1.809215H5.818182C5.929763-1.809215 6.105106-1.809215 6.105106-1.992528S5.929763-2.175841 5.818182-2.175841H3.474969V-4.527024C3.474969-4.638605 3.474969-4.813948 3.291656-4.813948S3.108344-4.638605 3.108344-4.527024V-2.175841H.757161C.645579-2.175841 .470237-2.175841 .470237-1.992528S.645579-1.809215 .757161-1.809215H3.108344V.541968C3.108344 .653549 3.108344 .828892 3.291656 .828892S3.474969 .653549 3.474969 .541968V-1.809215Z'/> +<path id='g5-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g6-40' d='M3.610908 2.618181C3.610908 2.585454 3.610908 2.563636 3.425454 2.378181C2.061818 1.003636 1.712727-1.058182 1.712727-2.727272C1.712727-4.625453 2.127272-6.523635 3.46909-7.887271C3.610908-8.01818 3.610908-8.039998 3.610908-8.072725C3.610908-8.149089 3.567272-8.181816 3.501817-8.181816C3.392726-8.181816 2.410908-7.439998 1.767272-6.054544C1.210909-4.854544 1.08-3.643635 1.08-2.727272C1.08-1.876363 1.2-.556363 1.799999 .676363C2.454545 2.018181 3.392726 2.727272 3.501817 2.727272C3.567272 2.727272 3.610908 2.694545 3.610908 2.618181Z'/> +<path id='g6-41' d='M3.152726-2.727272C3.152726-3.578181 3.032726-4.89818 2.432727-6.130907C1.778181-7.472725 .84-8.181816 .730909-8.181816C.665454-8.181816 .621818-8.13818 .621818-8.072725C.621818-8.039998 .621818-8.01818 .829091-7.821816C1.898181-6.741816 2.519999-5.007271 2.519999-2.727272C2.519999-.861818 2.116363 1.058182 .763636 2.432727C.621818 2.563636 .621818 2.585454 .621818 2.618181C.621818 2.683636 .665454 2.727272 .730909 2.727272C.84 2.727272 1.821818 1.985454 2.465454 .6C3.021817-.6 3.152726-1.810909 3.152726-2.727272Z'/> +<path id='g6-61' d='M7.494543-3.567272C7.65818-3.567272 7.865452-3.567272 7.865452-3.785453S7.65818-4.003635 7.505452-4.003635H.970909C.818182-4.003635 .610909-4.003635 .610909-3.785453S.818182-3.567272 .981818-3.567272H7.494543ZM7.505452-1.450909C7.65818-1.450909 7.865452-1.450909 7.865452-1.66909S7.65818-1.887272 7.494543-1.887272H.981818C.818182-1.887272 .610909-1.887272 .610909-1.66909S.818182-1.450909 .970909-1.450909H7.505452Z'/> +<path id='g3-27' d='M5.650908-4.06909C5.792726-4.06909 6.185453-4.06909 6.185453-4.439999C6.185453-4.701817 5.956362-4.701817 5.759998-4.701817H3.272726C1.625454-4.701817 .414545-2.901817 .414545-1.603636C.414545-.643636 1.058182 .12 2.050909 .12C3.338181 .12 4.78909-1.2 4.78909-2.879999C4.78909-3.065454 4.78909-3.58909 4.450908-4.06909H5.650908ZM2.061818-.12C1.527272-.12 1.090909-.512727 1.090909-1.298181C1.090909-1.625454 1.221818-2.519999 1.603636-3.163635C2.061818-3.916363 2.716363-4.06909 3.087272-4.06909C4.003635-4.06909 4.090908-3.34909 4.090908-3.010908C4.090908-2.498181 3.872726-1.603636 3.501817-1.047272C3.076363-.403636 2.487272-.12 2.061818-.12Z'/> +<path id='g3-122' d='M1.450909-.905454C2.039999-1.538181 2.356363-1.810909 2.74909-2.14909C2.74909-2.159999 3.425454-2.738181 3.818181-3.130908C4.854544-4.145453 5.094544-4.66909 5.094544-4.712726C5.094544-4.821817 4.996362-4.821817 4.974544-4.821817C4.89818-4.821817 4.865453-4.799999 4.810908-4.701817C4.483635-4.178181 4.254544-4.003635 3.992726-4.003635S3.599999-4.167272 3.436363-4.352726C3.22909-4.603635 3.043636-4.821817 2.683636-4.821817C1.865454-4.821817 1.363636-3.807272 1.363636-3.578181C1.363636-3.523635 1.396363-3.458181 1.494545-3.458181S1.614545-3.512726 1.636363-3.578181C1.843636-4.079999 2.476363-4.090908 2.563636-4.090908C2.792726-4.090908 2.999999-4.014544 3.250908-3.927272C3.687272-3.763635 3.807272-3.763635 4.090908-3.763635C3.698181-3.294545 2.781817-2.50909 2.574545-2.334545L1.592727-1.418181C.850909-.687273 .469091-.065455 .469091 .010909C.469091 .12 .578182 .12 .6 .12C.687273 .12 .709091 .098182 .774545-.021818C1.025454-.403636 1.352727-.698182 1.701818-.698182C1.952727-.698182 2.061818-.6 2.334545-.283636C2.519999-.054545 2.716363 .12 3.032726 .12C4.112726 .12 4.745453-1.265454 4.745453-1.56C4.745453-1.614545 4.701817-1.66909 4.614544-1.66909C4.516362-1.66909 4.494544-1.603636 4.461817-1.527272C4.210908-.818182 3.512726-.610909 3.152726-.610909C2.934545-.610909 2.738181-.676363 2.50909-.752727C2.138181-.894545 1.974545-.938182 1.745454-.938182C1.723636-.938182 1.54909-.938182 1.450909-.905454Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -66.69976)'> +<use x='70.734745' y='68.742217' xlink:href='#g3-27'/> +<use x='77.35975' y='68.742217' xlink:href='#g6-40'/> +<use x='81.602189' y='68.742217' xlink:href='#g3-122'/> +<use x='87.155241' y='64.783399' xlink:href='#g5-40'/> +<use x='90.448494' y='64.783399' xlink:href='#g2-105'/> +<use x='93.331633' y='64.783399' xlink:href='#g5-41'/> +<use x='97.123019' y='68.742217' xlink:href='#g6-41'/> +<use x='104.395704' y='68.742217' xlink:href='#g6-61'/> +<use x='131.848984' y='64.446968' xlink:href='#g5-49'/> +<rect x='117.106341' y='65.79678' height='.436352' width='33.719454'/> +<use x='117.106341' y='75.559745' xlink:href='#g5-49'/> +<use x='121.340524' y='75.559745' xlink:href='#g5-43'/> +<use x='127.927031' y='75.559745' xlink:href='#g2-101'/> +<use x='131.855415' y='73.290484' xlink:href='#g0-0'/> +<use x='137.611568' y='73.290484' xlink:href='#g1-122'/> +<use x='141.409796' y='71.152672' xlink:href='#g4-40'/> +<use x='144.287858' y='71.152672' xlink:href='#g1-105'/> +<use x='146.951484' y='71.152672' xlink:href='#g4-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/lr-target.svg b/_resources/lr-target.svg new file mode 100644 index 0000000..781a1df --- /dev/null +++ b/_resources/lr-target.svg @@ -0,0 +1,24 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='50.434337pt' height='13.755224pt' viewBox='-.299738 -.24919 50.434337 13.755224'> +<defs> +<path id='g0-25' d='M7.876361-4.930908C7.876361-5.159999 7.799998-5.25818 7.723634-5.25818C7.679998-5.25818 7.581816-5.214544 7.570907-4.963635C7.527271-4.210908 6.763634-3.763635 6.054544-3.763635C5.421817-3.763635 4.876362-4.101817 4.30909-4.483635C3.719999-4.876362 3.130908-5.269089 2.421818-5.269089C1.407272-5.269089 .6-4.494544 .6-3.490908C.6-3.250908 .687273-3.163635 .752727-3.163635C.861818-3.163635 .905454-3.370908 .905454-3.414545C.96-4.330908 1.854545-4.658181 2.421818-4.658181C3.054545-4.658181 3.599999-4.319999 4.167272-3.938181C4.756362-3.545454 5.345453-3.152726 6.054544-3.152726C7.069089-3.152726 7.876361-3.927272 7.876361-4.930908ZM7.876361-2.378181C7.876361-2.694545 7.745452-2.716363 7.723634-2.716363C7.679998-2.716363 7.581816-2.661817 7.570907-2.421818C7.527271-1.66909 6.763634-1.221818 6.054544-1.221818C5.421817-1.221818 4.876362-1.56 4.30909-1.941818C3.719999-2.334545 3.130908-2.727272 2.421818-2.727272C1.407272-2.727272 .6-1.952727 .6-.949091C.6-.709091 .687273-.621818 .752727-.621818C.861818-.621818 .905454-.829091 .905454-.872727C.96-1.78909 1.854545-2.116363 2.421818-2.116363C3.054545-2.116363 3.599999-1.778181 4.167272-1.396363C4.756362-1.003636 5.345453-.610909 6.054544-.610909C7.090907-.610909 7.876361-1.418181 7.876361-2.378181Z'/> +<path id='g1-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g3-40' d='M2.654047 1.992528C2.717808 1.992528 2.81345 1.992528 2.81345 1.896887C2.81345 1.865006 2.805479 1.857036 2.701868 1.753425C1.609963 .72528 1.338979-.757161 1.338979-1.992528C1.338979-4.28792 2.287422-5.363885 2.693898-5.730511C2.805479-5.834122 2.81345-5.842092 2.81345-5.881943S2.781569-5.977584 2.701868-5.977584C2.574346-5.977584 2.175841-5.571108 2.11208-5.499377C1.044085-4.383562 .820922-2.948941 .820922-1.992528C.820922-.207223 1.570112 1.227397 2.654047 1.992528Z'/> +<path id='g3-41' d='M2.462765-1.992528C2.462765-2.749689 2.335243-3.658281 1.841096-4.598755C1.45056-5.332005 .72528-5.977584 .581818-5.977584C.502117-5.977584 .478207-5.921793 .478207-5.881943C.478207-5.850062 .478207-5.834122 .573848-5.738481C1.689664-4.678456 1.944707-3.219925 1.944707-1.992528C1.944707 .294894 .996264 1.378829 .589788 1.745455C.486177 1.849066 .478207 1.857036 .478207 1.896887S.502117 1.992528 .581818 1.992528C.70934 1.992528 1.107846 1.586052 1.171606 1.514321C2.239601 .398506 2.462765-1.036115 2.462765-1.992528Z'/> +<path id='g2-121' d='M5.301817-4.156362C5.345453-4.30909 5.345453-4.330908 5.345453-4.407271C5.345453-4.603635 5.192726-4.701817 5.02909-4.701817C4.919999-4.701817 4.745453-4.636362 4.647271-4.472726C4.625453-4.418181 4.538181-4.079999 4.494544-3.883635C4.418181-3.599999 4.341817-3.305454 4.276362-3.010908L3.785453-1.047272C3.741817-.883636 3.272726-.12 2.552727-.12C1.996363-.12 1.876363-.6 1.876363-1.003636C1.876363-1.505454 2.061818-2.181818 2.432727-3.141817C2.607272-3.58909 2.650908-3.70909 2.650908-3.927272C2.650908-4.418181 2.301818-4.821817 1.756363-4.821817C.72-4.821817 .316364-3.239999 .316364-3.141817C.316364-3.032726 .425454-3.032726 .447273-3.032726C.556363-3.032726 .567273-3.054545 .621818-3.22909C.916363-4.254544 1.352727-4.581817 1.723636-4.581817C1.810909-4.581817 1.996363-4.581817 1.996363-4.232726C1.996363-3.959999 1.887272-3.676363 1.810909-3.46909C1.374545-2.312727 1.178181-1.690909 1.178181-1.178181C1.178181-.207273 1.865454 .12 2.50909 .12C2.934545 .12 3.305454-.065455 3.610908-.370909C3.46909 .196364 3.338181 .730909 2.901817 1.309091C2.618181 1.68 2.203636 1.996363 1.701818 1.996363C1.54909 1.996363 1.058182 1.963636 .872727 1.538181C1.047272 1.538181 1.189091 1.538181 1.341818 1.407272C1.450909 1.309091 1.56 1.167272 1.56 .96C1.56 .621818 1.265454 .578182 1.156363 .578182C.905454 .578182 .545454 .752727 .545454 1.287272C.545454 1.832727 1.025454 2.236363 1.701818 2.236363C2.825454 2.236363 3.94909 1.243636 4.254544 .010909L5.301817-4.156362Z'/> +<path id='g4-94' d='M2.727272-7.570907L1.265454-6.087271L1.461818-5.890907L2.727272-7.003634L3.981817-5.890907L4.178181-6.087271L2.727272-7.570907Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -66.69976)'> +<use x='71.483493' y='68.742217' xlink:href='#g4-94'/> +<use x='70.734745' y='68.742217' xlink:href='#g2-121'/> +<use x='76.474675' y='64.783399' xlink:href='#g3-40'/> +<use x='79.767928' y='64.783399' xlink:href='#g1-105'/> +<use x='82.651068' y='64.783399' xlink:href='#g3-41'/> +<use x='89.47269' y='68.742217' xlink:href='#g0-25'/> +<use x='100.987813' y='68.742217' xlink:href='#g2-121'/> +<use x='106.727744' y='64.783399' xlink:href='#g3-40'/> +<use x='110.020997' y='64.783399' xlink:href='#g1-105'/> +<use x='112.904136' y='64.783399' xlink:href='#g3-41'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/metric-clf-error-weighted.svg b/_resources/metric-clf-error-weighted.svg new file mode 100644 index 0000000..7f545af --- /dev/null +++ b/_resources/metric-clf-error-weighted.svg @@ -0,0 +1,61 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='162.370552pt' height='36.972014pt' viewBox='-.244117 -.232683 162.370552 36.972014'> +<defs> +<path id='g1-54' d='M7.531756-8.093649C7.627397-8.261021 7.627397-8.284932 7.627397-8.320797C7.627397-8.404483 7.555666-8.5599 7.388294-8.5599C7.244832-8.5599 7.208966-8.488169 7.12528-8.320797L1.75741 2.116065C1.661768 2.283437 1.661768 2.307347 1.661768 2.343213C1.661768 2.438854 1.745455 2.582316 1.900872 2.582316C2.044334 2.582316 2.080199 2.510585 2.163885 2.343213L7.531756-8.093649Z'/> +<path id='g1-76' d='M2.15193-1.111831C2.797509-2.116065 3.000747-2.881196 3.156164-3.514819C3.574595-5.164633 4.028892-6.599253 4.770112-7.424159C4.913574-7.579577 5.009215-7.687173 5.391781-7.687173C6.216687-7.687173 6.240598-6.862267 6.240598-6.694894C6.240598-6.479701 6.180822-6.312329 6.180822-6.252553C6.180822-6.168867 6.252553-6.168867 6.264508-6.168867C6.455791-6.168867 6.77858-6.300374 7.07746-6.515567C7.292653-6.682939 7.400249-6.802491 7.400249-7.292653C7.400249-7.938232 7.065504-8.428394 6.396015-8.428394C6.01345-8.428394 4.961395-8.332752 3.789788-7.149191C2.833375-6.168867 2.271482-4.016936 2.044334-3.120299C1.829141-2.295392 1.733499-1.924782 1.374844-1.207472C1.291158-1.06401 .980324-.537983 .812951-.382565C.490162-.083686 .37061 .131507 .37061 .191283C.37061 .215193 .394521 .263014 .478207 .263014C.526027 .263014 .777086 .215193 1.08792 .011955C1.291158-.107597 1.315068-.131507 1.590037-.418431C2.187796-.406476 2.606227-.298879 3.359402-.083686C3.969116 .083686 4.578829 .263014 5.188543 .263014C6.156912 .263014 7.137235-.466252 7.519801-.992279C7.758904-1.315068 7.830635-1.613948 7.830635-1.649813C7.830635-1.733499 7.758904-1.733499 7.746949-1.733499C7.555666-1.733499 7.268742-1.601993 7.065504-1.458531C6.742715-1.255293 6.718804-1.183562 6.647073-.980324C6.587298-.789041 6.515567-.6934 6.467746-.621669C6.372105-.478207 6.360149-.478207 6.180822-.478207C5.606974-.478207 5.009215-.657534 4.220174-.872727C3.88543-.968369 3.227895-1.159651 2.630137-1.159651C2.47472-1.159651 2.307347-1.147696 2.15193-1.111831Z'/> +<path id='g1-102' d='M3.383313-7.376339C3.383313-7.854545 3.694147-8.619676 4.99726-8.703362C5.057036-8.715318 5.104857-8.763138 5.104857-8.834869C5.104857-8.966376 5.009215-8.966376 4.877709-8.966376C3.682192-8.966376 2.594271-8.356663 2.582316-7.47198V-4.746202C2.582316-4.27995 2.582316-3.897385 2.10411-3.502864C1.685679-3.156164 1.231382-3.132254 .968369-3.120299C.908593-3.108344 .860772-3.060523 .860772-2.988792C.860772-2.86924 .932503-2.86924 1.052055-2.857285C1.841096-2.809465 2.414944-2.379078 2.546451-1.793275C2.582316-1.661768 2.582316-1.637858 2.582316-1.207472V1.159651C2.582316 1.661768 2.582316 2.044334 3.156164 2.49863C3.622416 2.857285 4.411457 2.988792 4.877709 2.988792C5.009215 2.988792 5.104857 2.988792 5.104857 2.857285C5.104857 2.737733 5.033126 2.737733 4.913574 2.725778C4.160399 2.677958 3.574595 2.295392 3.419178 1.685679C3.383313 1.578082 3.383313 1.554172 3.383313 1.123786V-1.3868C3.383313-1.936737 3.287671-2.139975 2.905106-2.52254C2.654047-2.773599 2.307347-2.893151 1.972603-2.988792C2.952927-3.263761 3.383313-3.813699 3.383313-4.507098V-7.376339Z'/> +<path id='g1-103' d='M2.582316 1.398755C2.582316 1.876961 2.271482 2.642092 .968369 2.725778C.908593 2.737733 .860772 2.785554 .860772 2.857285C.860772 2.988792 .992279 2.988792 1.099875 2.988792C2.259527 2.988792 3.371357 2.402989 3.383313 1.494396V-1.231382C3.383313-1.697634 3.383313-2.080199 3.861519-2.47472C4.27995-2.82142 4.734247-2.84533 4.99726-2.857285C5.057036-2.86924 5.104857-2.917061 5.104857-2.988792C5.104857-3.108344 5.033126-3.108344 4.913574-3.120299C4.124533-3.16812 3.550685-3.598506 3.419178-4.184309C3.383313-4.315816 3.383313-4.339726 3.383313-4.770112V-7.137235C3.383313-7.639352 3.383313-8.021918 2.809465-8.476214C2.331258-8.846824 1.506351-8.966376 1.099875-8.966376C.992279-8.966376 .860772-8.966376 .860772-8.834869C.860772-8.715318 .932503-8.715318 1.052055-8.703362C1.80523-8.655542 2.391034-8.272976 2.546451-7.663263C2.582316-7.555666 2.582316-7.531756 2.582316-7.10137V-4.590785C2.582316-4.040847 2.677958-3.837609 3.060523-3.455044C3.311582-3.203985 3.658281-3.084433 3.993026-2.988792C3.012702-2.713823 2.582316-2.163885 2.582316-1.470486V1.398755Z'/> +<path id='g2-100' d='M3.616438-3.969116C3.622416-3.993026 3.634371-4.028892 3.634371-4.05878C3.634371-4.154421 3.514819-4.148443 3.443088-4.142466L2.773599-4.088667C2.67198-4.08269 2.594271-4.076712 2.594271-3.939228C2.594271-3.843587 2.67198-3.843587 2.761644-3.843587C2.940971-3.843587 2.982814-3.831631 3.060523-3.801743C3.054545-3.71208 3.054545-3.700125 3.036613-3.622416C2.911083-3.108344 2.815442-2.707846 2.69589-2.27746C2.612204-2.414944 2.408966-2.636115 2.038356-2.636115C1.273225-2.636115 .448319-1.835118 .448319-.956413C.448319-.310834 .902615 .059776 1.41071 .059776C1.811208 .059776 2.15193-.215193 2.30137-.364633C2.414944 .011955 2.815442 .059776 2.946949 .059776C3.162142 .059776 3.317559-.059776 3.431133-.245081C3.580573-.484184 3.664259-.830884 3.664259-.860772C3.664259-.872727 3.658281-.944458 3.550685-.944458C3.461021-.944458 3.449066-.902615 3.425156-.806974C3.329514-.442341 3.203985-.137484 2.970859-.137484C2.767621-.137484 2.749689-.352677 2.749689-.442341C2.749689-.52005 2.749689-.537983 2.779577-.645579L3.616438-3.969116ZM2.32528-.783064C2.295392-.675467 2.295392-.663512 2.211706-.573848C1.882939-.203238 1.578082-.137484 1.428643-.137484C1.189539-.137484 .956413-.298879 .956413-.723288C.956413-.968369 1.081943-1.554172 1.273225-1.894894C1.452553-2.217684 1.75741-2.438854 2.044334-2.438854C2.492653-2.438854 2.606227-1.966625 2.606227-1.924782L2.588294-1.841096L2.32528-.783064Z'/> +<path id='g2-101' d='M1.321046-1.344956C1.715567-1.344956 2.169863-1.374844 2.480697-1.476463C3.000747-1.649813 3.024658-2.014446 3.024658-2.10411C3.024658-2.397011 2.737733-2.636115 2.229639-2.636115C1.494396-2.636115 .478207-2.110087 .478207-1.08792C.478207-.454296 .89066 .059776 1.637858 .059776C2.749689 .059776 3.186052-.532005 3.186052-.603736C3.186052-.657534 3.120299-.723288 3.072478-.723288C3.04259-.723288 3.036613-.71731 2.946949-.627646C2.588294-.221171 1.97858-.137484 1.649813-.137484C1.165629-.137484 .992279-.484184 .992279-.86675C.992279-.91457 .998257-1.111831 1.058032-1.344956H1.321046ZM1.111831-1.542217C1.309091-2.247572 1.894894-2.438854 2.229639-2.438854C2.510585-2.438854 2.749689-2.313325 2.749689-2.098132C2.749689-1.542217 1.751432-1.542217 1.111831-1.542217Z'/> +<path id='g2-118' d='M3.425156-2.145953C3.425156-2.576339 3.180075-2.642092 3.102366-2.642092C2.929016-2.642092 2.767621-2.456787 2.767621-2.313325C2.767621-2.199751 2.84533-2.139975 2.881196-2.110087C3.000747-2.020423 3.108344-1.876961 3.108344-1.649813C3.108344-1.338979 2.677958-.137484 1.97858-.137484C1.458531-.137484 1.458531-.555915 1.458531-.669489C1.458531-.926526 1.542217-1.195517 1.799253-1.835118C1.847073-1.960648 1.865006-2.008468 1.865006-2.116065C1.865006-2.450809 1.560149-2.636115 1.267248-2.636115C.657534-2.636115 .364633-1.841096 .364633-1.715567C.364633-1.685679 .388543-1.63188 .472229-1.63188S.573848-1.667746 .591781-1.721544C.759153-2.289415 1.069988-2.438854 1.243337-2.438854C1.362889-2.438854 1.404732-2.361146 1.404732-2.223661C1.404732-2.098132 1.327024-1.894894 1.26127-1.733499C.986301-1.046077 .986301-.86675 .986301-.74122C.986301-.101619 1.530262 .059776 1.95467 .059776C2.952927 .059776 3.425156-1.554172 3.425156-2.145953Z'/> +<path id='g5-40' d='M2.654047 1.992528C2.717808 1.992528 2.81345 1.992528 2.81345 1.896887C2.81345 1.865006 2.805479 1.857036 2.701868 1.753425C1.609963 .72528 1.338979-.757161 1.338979-1.992528C1.338979-4.28792 2.287422-5.363885 2.693898-5.730511C2.805479-5.834122 2.81345-5.842092 2.81345-5.881943S2.781569-5.977584 2.701868-5.977584C2.574346-5.977584 2.175841-5.571108 2.11208-5.499377C1.044085-4.383562 .820922-2.948941 .820922-1.992528C.820922-.207223 1.570112 1.227397 2.654047 1.992528Z'/> +<path id='g5-41' d='M2.462765-1.992528C2.462765-2.749689 2.335243-3.658281 1.841096-4.598755C1.45056-5.332005 .72528-5.977584 .581818-5.977584C.502117-5.977584 .478207-5.921793 .478207-5.881943C.478207-5.850062 .478207-5.834122 .573848-5.738481C1.689664-4.678456 1.944707-3.219925 1.944707-1.992528C1.944707 .294894 .996264 1.378829 .589788 1.745455C.486177 1.849066 .478207 1.857036 .478207 1.896887S.502117 1.992528 .581818 1.992528C.70934 1.992528 1.107846 1.586052 1.171606 1.514321C2.239601 .398506 2.462765-1.036115 2.462765-1.992528Z'/> +<path id='g5-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g5-61' d='M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z'/> +<path id='g3-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g3-109' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.745455-1.928767 1.793275-2.12005 1.809215-2.199751C1.825156-2.239601 2.088169-2.757659 2.438854-3.020672C2.709838-3.227895 2.972852-3.291656 3.196015-3.291656C3.490909-3.291656 3.650311-3.116314 3.650311-2.749689C3.650311-2.558406 3.602491-2.375093 3.514819-2.016438C3.459029-1.809215 3.323537-1.275218 3.275716-1.060025L3.156164-.581818C3.116314-.446326 3.060523-.207223 3.060523-.167372C3.060523 .01594 3.211955 .079701 3.315567 .079701C3.459029 .079701 3.57858-.01594 3.634371-.111582C3.658281-.159402 3.722042-.430386 3.761893-.597758L3.945205-1.307098C3.969116-1.42665 4.048817-1.729514 4.072727-1.849066C4.184309-2.279452 4.184309-2.287422 4.367621-2.550436C4.630635-2.940971 5.00523-3.291656 5.539228-3.291656C5.826152-3.291656 5.993524-3.124284 5.993524-2.749689C5.993524-2.311333 5.65878-1.39477 5.507347-1.012204C5.427646-.804981 5.403736-.749191 5.403736-.597758C5.403736-.143462 5.778331 .079701 6.121046 .079701C6.902117 .079701 7.228892-1.036115 7.228892-1.139726C7.228892-1.219427 7.165131-1.243337 7.10934-1.243337C7.013699-1.243337 6.997758-1.187547 6.973848-1.107846C6.782565-.446326 6.447821-.143462 6.144956-.143462C6.017435-.143462 5.953674-.223163 5.953674-.406476S6.017435-.765131 6.097136-.964384C6.216687-1.267248 6.567372-2.183811 6.567372-2.630137C6.567372-3.227895 6.152927-3.514819 5.579078-3.514819C5.029141-3.514819 4.574844-3.227895 4.216189-2.733748C4.152428-3.371357 3.642341-3.514819 3.227895-3.514819C2.86127-3.514819 2.375093-3.387298 1.936737-2.81345C1.880946-3.291656 1.498381-3.514819 1.123786-3.514819C.844832-3.514819 .645579-3.347447 .510087-3.076463C.318804-2.701868 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g0-80' d='M5.033126 6.38406L.789041 11.632379C.6934 11.75193 .681445 11.775841 .681445 11.823661C.681445 11.955168 .789041 11.955168 1.004234 11.955168H10.915068L11.943213 8.978331H11.644334C11.345455 9.874969 10.544458 10.604234 9.528269 10.950934C9.336986 11.01071 8.51208 11.297634 6.75467 11.297634H1.673724L5.822167 6.168867C5.905853 6.06127 5.929763 6.025405 5.929763 5.977584S5.917808 5.917808 5.846077 5.810212L1.960648 .478207H6.694894C8.057783 .478207 10.807472 .561893 11.644334 2.797509H11.943213L10.915068 0H1.004234C.681445 0 .669489 .011955 .669489 .382565L5.033126 6.38406Z'/> +<path id='g0-88' d='M15.135243 16.737235L16.581818 12.911582H16.282939C15.816687 14.154919 14.54944 14.96787 13.174595 15.326526C12.923537 15.386301 11.75193 15.697136 9.456538 15.697136H2.247572L8.332752 8.5599C8.416438 8.464259 8.440349 8.428394 8.440349 8.368618C8.440349 8.344707 8.440349 8.308842 8.356663 8.18929L2.785554 .573848H9.336986C10.938979 .573848 12.026899 .74122 12.134496 .765131C12.780075 .860772 13.820174 1.06401 14.764633 1.661768C15.063512 1.853051 15.876463 2.391034 16.282939 3.359402H16.581818L15.135243 0H1.004234C.729265 0 .71731 .011955 .681445 .083686C.669489 .119552 .669489 .3467 .669489 .478207L6.993773 9.133748L.800996 16.390535C.681445 16.533998 .681445 16.593773 .681445 16.605729C.681445 16.737235 .789041 16.737235 1.004234 16.737235H15.135243Z'/> +<path id='g4-119' d='M4.112578-.729265C4.375592-.011955 5.116812 .119552 5.571108 .119552C6.479701 .119552 7.017684-.669489 7.352428-1.494396C7.627397-2.187796 8.069738-3.765878 8.069738-4.471233C8.069738-5.200498 7.699128-5.272229 7.591532-5.272229C7.304608-5.272229 7.041594-4.985305 7.041594-4.746202C7.041594-4.60274 7.12528-4.519054 7.185056-4.471233C7.292653-4.363636 7.603487-4.040847 7.603487-3.419178C7.603487-3.012702 7.268742-1.888917 7.017684-1.327024C6.682939-.597758 6.240598-.119552 5.618929-.119552C4.94944-.119552 4.734247-.621669 4.734247-1.171606C4.734247-1.518306 4.841843-1.936737 4.889664-2.139975L5.391781-4.148443C5.451557-4.387547 5.559153-4.805978 5.559153-4.853798C5.559153-5.033126 5.415691-5.152677 5.236364-5.152677C4.889664-5.152677 4.805978-4.853798 4.734247-4.566874C4.614695-4.100623 4.112578-2.080199 4.064757-1.817186C4.016936-1.613948 4.016936-1.482441 4.016936-1.219427C4.016936-.932503 3.634371-.478207 3.622416-.454296C3.490909-.32279 3.299626-.119552 2.929016-.119552C1.984558-.119552 1.984558-1.016189 1.984558-1.219427C1.984558-1.601993 2.068244-2.12802 2.606227-3.550685C2.749689-3.921295 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.920548-4.60274 1.3868-5.033126 1.829141-5.033126C1.948692-5.033126 2.139975-5.021171 2.139975-4.638605C2.139975-4.578829 2.139975-4.327771 1.936737-3.801743C1.374844-2.307347 1.255293-1.817186 1.255293-1.362889C1.255293-.107597 2.283437 .119552 2.893151 .119552C3.096389 .119552 3.634371 .119552 4.112578-.729265Z'/> +<path id='g4-121' d='M3.144209 1.338979C2.82142 1.793275 2.355168 2.199751 1.769365 2.199751C1.625903 2.199751 1.052055 2.175841 .872727 1.625903C.908593 1.637858 .968369 1.637858 .992279 1.637858C1.350934 1.637858 1.590037 1.327024 1.590037 1.052055S1.362889 .681445 1.183562 .681445C.992279 .681445 .573848 .824907 .573848 1.41071C.573848 2.020423 1.08792 2.438854 1.769365 2.438854C2.964882 2.438854 4.172354 1.338979 4.507098 .011955L5.678705-4.65056C5.69066-4.710336 5.71457-4.782067 5.71457-4.853798C5.71457-5.033126 5.571108-5.152677 5.391781-5.152677C5.284184-5.152677 5.033126-5.104857 4.937484-4.746202L4.052802-1.231382C3.993026-1.016189 3.993026-.992279 3.897385-.860772C3.658281-.526027 3.263761-.119552 2.689913-.119552C2.020423-.119552 1.960648-.777086 1.960648-1.099875C1.960648-1.78132 2.283437-2.701868 2.606227-3.56264C2.737733-3.90934 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.033126 2.139975-4.638605C2.139975-4.327771 2.008468-3.981071 1.829141-3.526775C1.243337-1.960648 1.243337-1.566127 1.243337-1.279203C1.243337-.143462 2.056289 .119552 2.654047 .119552C3.000747 .119552 3.431133 .011955 3.849564-.430386L3.861519-.418431C3.682192 .286924 3.56264 .753176 3.144209 1.338979Z'/> +<path id='g6-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +<path id='g6-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g6-94' d='M2.929016-8.296887L1.362889-6.670984L1.554172-6.491656L2.917061-7.723039L4.291905-6.491656L4.483188-6.670984L2.929016-8.296887Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -65.34204 -62.281577)'> +<use x='71.316882' y='65.624772' xlink:href='#g6-49'/> +<rect x='57.608781' y='70.484645' height='.478187' width='33.269192'/> +<use x='57.608781' y='72.946741' xlink:href='#g0-80'/> +<use x='72.220658' y='81.913193' xlink:href='#g4-119'/> +<use x='80.910198' y='78.459485' xlink:href='#g5-40'/> +<use x='84.203452' y='78.459485' xlink:href='#g3-105'/> +<use x='87.086591' y='78.459485' xlink:href='#g5-41'/> +<use x='94.065984' y='58.425345' xlink:href='#g3-109'/> +<use x='101.55651' y='59.83119' xlink:href='#g2-100'/> +<use x='105.396965' y='59.83119' xlink:href='#g2-101'/> +<use x='108.917789' y='59.83119' xlink:href='#g2-118'/> +<use x='95.081843' y='62.355066' xlink:href='#g0-88'/> +<use x='96.864237' y='87.549421' xlink:href='#g3-105'/> +<use x='99.747377' y='87.549421' xlink:href='#g5-61'/> +<use x='106.333883' y='87.549421' xlink:href='#g5-49'/> +<use x='115.358817' y='73.712531' xlink:href='#g4-119'/> +<use x='124.048357' y='68.776345' xlink:href='#g5-40'/> +<use x='127.34161' y='68.776345' xlink:href='#g3-105'/> +<use x='130.22475' y='68.776345' xlink:href='#g5-41'/> +<use x='134.016135' y='73.712531' xlink:href='#g1-76'/> +<use x='142.261913' y='73.712531' xlink:href='#g1-102'/> +<use x='149.031684' y='73.712531' xlink:href='#g6-94'/> +<use x='148.23952' y='73.712531' xlink:href='#g4-121'/> +<use x='154.376171' y='68.776345' xlink:href='#g5-40'/> +<use x='157.669425' y='68.776345' xlink:href='#g3-105'/> +<use x='160.552564' y='68.776345' xlink:href='#g5-41'/> +<use x='167.664768' y='73.712531' xlink:href='#g1-54'/> +<use x='167.664768' y='73.712531' xlink:href='#g6-61'/> +<use x='180.090248' y='73.712531' xlink:href='#g4-121'/> +<use x='186.2269' y='68.776345' xlink:href='#g5-40'/> +<use x='189.520153' y='68.776345' xlink:href='#g3-105'/> +<use x='192.403293' y='68.776345' xlink:href='#g5-41'/> +<use x='196.194678' y='73.712531' xlink:href='#g1-103'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/metric-clf-error-weights.svg b/_resources/metric-clf-error-weights.svg new file mode 100644 index 0000000..6cbd1ed --- /dev/null +++ b/_resources/metric-clf-error-weights.svg @@ -0,0 +1,86 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='237.458334pt' height='32.408907pt' viewBox='-.239051 -.228392 237.458334 32.408907'> +<defs> +<path id='g0-26' d='M4.97335 17.801245C4.97335 17.239352 4.97335 16.35467 4.220174 15.386301C3.753923 14.788543 3.072478 14.238605 2.247572 13.867995C4.578829 12.744209 4.97335 11.022665 4.97335 10.34122V3.455044C4.97335 2.713823 4.97335 1.183562 7.292653 .035866C7.388294-.011955 7.388294-.035866 7.388294-.215193C7.388294-.466252 7.388294-.478207 7.12528-.478207C6.957908-.478207 6.933998-.478207 6.623163-.334745C5.439601 .251059 4.267995 1.195517 4.016936 2.654047C3.981071 2.893151 3.981071 3.000747 3.981071 3.801743V9.325031C3.981071 9.695641 3.981071 10.31731 3.969116 10.448817C3.861519 11.70411 3.132254 12.79203 1.769365 13.545205C1.578082 13.652802 1.566127 13.664757 1.566127 13.85604C1.566127 14.059278 1.578082 14.071233 1.745455 14.166874C2.546451 14.609215 3.694147 15.469988 3.93325 16.964384C3.981071 17.239352 3.981071 17.263263 3.981071 17.39477V24.496139C3.981071 26.229639 5.176588 27.36538 6.659029 28.070735C6.922042 28.202242 6.945953 28.202242 7.12528 28.202242C7.376339 28.202242 7.388294 28.202242 7.388294 27.939228C7.388294 27.747945 7.376339 27.73599 7.280697 27.676214C6.790535 27.437111 5.248319 26.660025 4.99726 24.938481C4.97335 24.771108 4.97335 24.639601 4.97335 23.922291V17.801245Z'/> +<path id='g4-40' d='M3.88543 2.905106C3.88543 2.86924 3.88543 2.84533 3.682192 2.642092C2.486675 1.43462 1.817186-.537983 1.817186-2.976837C1.817186-5.296139 2.379078-7.292653 3.765878-8.703362C3.88543-8.810959 3.88543-8.834869 3.88543-8.870735C3.88543-8.942466 3.825654-8.966376 3.777833-8.966376C3.622416-8.966376 2.642092-8.105604 2.056289-6.933998C1.446575-5.726526 1.171606-4.447323 1.171606-2.976837C1.171606-1.912827 1.338979-.490162 1.960648 .789041C2.666002 2.223661 3.646326 3.000747 3.777833 3.000747C3.825654 3.000747 3.88543 2.976837 3.88543 2.905106Z'/> +<path id='g4-41' d='M3.371357-2.976837C3.371357-3.88543 3.251806-5.36787 2.582316-6.75467C1.876961-8.18929 .896638-8.966376 .765131-8.966376C.71731-8.966376 .657534-8.942466 .657534-8.870735C.657534-8.834869 .657534-8.810959 .860772-8.607721C2.056289-7.400249 2.725778-5.427646 2.725778-2.988792C2.725778-.669489 2.163885 1.327024 .777086 2.737733C.657534 2.84533 .657534 2.86924 .657534 2.905106C.657534 2.976837 .71731 3.000747 .765131 3.000747C.920548 3.000747 1.900872 2.139975 2.486675 .968369C3.096389-.251059 3.371357-1.542217 3.371357-2.976837Z'/> +<path id='g4-48' d='M5.355915-3.825654C5.355915-4.817933 5.296139-5.786301 4.865753-6.694894C4.375592-7.687173 3.514819-7.950187 2.929016-7.950187C2.235616-7.950187 1.3868-7.603487 .944458-6.611208C.609714-5.858032 .490162-5.116812 .490162-3.825654C.490162-2.666002 .573848-1.793275 1.004234-.944458C1.470486-.035866 2.295392 .251059 2.917061 .251059C3.957161 .251059 4.554919-.37061 4.901619-1.06401C5.332005-1.960648 5.355915-3.132254 5.355915-3.825654ZM2.917061 .011955C2.534496 .011955 1.75741-.203238 1.530262-1.506351C1.398755-2.223661 1.398755-3.132254 1.398755-3.969116C1.398755-4.94944 1.398755-5.834122 1.590037-6.539477C1.793275-7.340473 2.402989-7.711083 2.917061-7.711083C3.371357-7.711083 4.064757-7.436115 4.291905-6.40797C4.447323-5.726526 4.447323-4.782067 4.447323-3.969116C4.447323-3.16812 4.447323-2.259527 4.315816-1.530262C4.088667-.215193 3.335492 .011955 2.917061 .011955Z'/> +<path id='g4-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +<path id='g4-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g3-40' d='M2.654047 1.992528C2.717808 1.992528 2.81345 1.992528 2.81345 1.896887C2.81345 1.865006 2.805479 1.857036 2.701868 1.753425C1.609963 .72528 1.338979-.757161 1.338979-1.992528C1.338979-4.28792 2.287422-5.363885 2.693898-5.730511C2.805479-5.834122 2.81345-5.842092 2.81345-5.881943S2.781569-5.977584 2.701868-5.977584C2.574346-5.977584 2.175841-5.571108 2.11208-5.499377C1.044085-4.383562 .820922-2.948941 .820922-1.992528C.820922-.207223 1.570112 1.227397 2.654047 1.992528Z'/> +<path id='g3-41' d='M2.462765-1.992528C2.462765-2.749689 2.335243-3.658281 1.841096-4.598755C1.45056-5.332005 .72528-5.977584 .581818-5.977584C.502117-5.977584 .478207-5.921793 .478207-5.881943C.478207-5.850062 .478207-5.834122 .573848-5.738481C1.689664-4.678456 1.944707-3.219925 1.944707-1.992528C1.944707 .294894 .996264 1.378829 .589788 1.745455C.486177 1.849066 .478207 1.857036 .478207 1.896887S.502117 1.992528 .581818 1.992528C.70934 1.992528 1.107846 1.586052 1.171606 1.514321C2.239601 .398506 2.462765-1.036115 2.462765-1.992528Z'/> +<path id='g1-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g2-59' d='M2.331258 .047821C2.331258-.645579 2.10411-1.159651 1.613948-1.159651C1.231382-1.159651 1.0401-.848817 1.0401-.585803S1.219427 0 1.625903 0C1.78132 0 1.912827-.047821 2.020423-.155417C2.044334-.179328 2.056289-.179328 2.068244-.179328C2.092154-.179328 2.092154-.011955 2.092154 .047821C2.092154 .442341 2.020423 1.219427 1.327024 1.996513C1.195517 2.139975 1.195517 2.163885 1.195517 2.187796C1.195517 2.247572 1.255293 2.307347 1.315068 2.307347C1.41071 2.307347 2.331258 1.422665 2.331258 .047821Z'/> +<path id='g2-97' d='M3.598506-1.422665C3.53873-1.219427 3.53873-1.195517 3.371357-.968369C3.108344-.633624 2.582316-.119552 2.020423-.119552C1.530262-.119552 1.255293-.561893 1.255293-1.267248C1.255293-1.924782 1.625903-3.263761 1.853051-3.765878C2.259527-4.60274 2.82142-5.033126 3.287671-5.033126C4.076712-5.033126 4.23213-4.052802 4.23213-3.957161C4.23213-3.945205 4.196264-3.789788 4.184309-3.765878L3.598506-1.422665ZM4.363636-4.483188C4.23213-4.794022 3.90934-5.272229 3.287671-5.272229C1.936737-5.272229 .478207-3.526775 .478207-1.75741C.478207-.573848 1.171606 .119552 1.984558 .119552C2.642092 .119552 3.203985-.394521 3.53873-.789041C3.658281-.083686 4.220174 .119552 4.578829 .119552S5.224408-.095641 5.439601-.526027C5.630884-.932503 5.798257-1.661768 5.798257-1.709589C5.798257-1.769365 5.750436-1.817186 5.678705-1.817186C5.571108-1.817186 5.559153-1.75741 5.511333-1.578082C5.332005-.872727 5.104857-.119552 4.614695-.119552C4.267995-.119552 4.244085-.430386 4.244085-.669489C4.244085-.944458 4.27995-1.075965 4.387547-1.542217C4.471233-1.841096 4.531009-2.10411 4.62665-2.450809C5.068991-4.244085 5.176588-4.674471 5.176588-4.746202C5.176588-4.913574 5.045081-5.045081 4.865753-5.045081C4.483188-5.045081 4.387547-4.62665 4.363636-4.483188Z'/> +<path id='g2-99' d='M4.674471-4.495143C4.447323-4.495143 4.339726-4.495143 4.172354-4.351681C4.100623-4.291905 3.969116-4.112578 3.969116-3.921295C3.969116-3.682192 4.148443-3.53873 4.375592-3.53873C4.662516-3.53873 4.985305-3.777833 4.985305-4.25604C4.985305-4.829888 4.435367-5.272229 3.610461-5.272229C2.044334-5.272229 .478207-3.56264 .478207-1.865006C.478207-.824907 1.123786 .119552 2.343213 .119552C3.969116 .119552 4.99726-1.147696 4.99726-1.303113C4.99726-1.374844 4.925529-1.43462 4.877709-1.43462C4.841843-1.43462 4.829888-1.422665 4.722291-1.315068C3.957161-.298879 2.82142-.119552 2.367123-.119552C1.542217-.119552 1.279203-.836862 1.279203-1.43462C1.279203-1.853051 1.482441-3.012702 1.912827-3.825654C2.223661-4.387547 2.86924-5.033126 3.622416-5.033126C3.777833-5.033126 4.435367-5.009215 4.674471-4.495143Z'/> +<path id='g2-102' d='M5.332005-4.805978C5.571108-4.805978 5.66675-4.805978 5.66675-5.033126C5.66675-5.152677 5.571108-5.152677 5.355915-5.152677H4.387547C4.614695-6.38406 4.782067-7.232877 4.877709-7.615442C4.94944-7.902366 5.200498-8.177335 5.511333-8.177335C5.762391-8.177335 6.01345-8.069738 6.133001-7.962142C5.66675-7.914321 5.523288-7.567621 5.523288-7.364384C5.523288-7.12528 5.702615-6.981818 5.929763-6.981818C6.168867-6.981818 6.527522-7.185056 6.527522-7.639352C6.527522-8.141469 6.025405-8.416438 5.499377-8.416438C4.985305-8.416438 4.483188-8.033873 4.244085-7.567621C4.028892-7.149191 3.90934-6.718804 3.634371-5.152677H2.833375C2.606227-5.152677 2.486675-5.152677 2.486675-4.937484C2.486675-4.805978 2.558406-4.805978 2.797509-4.805978H3.56264C3.347447-3.694147 2.857285-.992279 2.582316 .286924C2.379078 1.327024 2.199751 2.199751 1.601993 2.199751C1.566127 2.199751 1.219427 2.199751 1.004234 1.972603C1.613948 1.924782 1.613948 1.398755 1.613948 1.3868C1.613948 1.147696 1.43462 1.004234 1.207472 1.004234C.968369 1.004234 .609714 1.207472 .609714 1.661768C.609714 2.175841 1.135741 2.438854 1.601993 2.438854C2.82142 2.438854 3.323537 .251059 3.455044-.3467C3.670237-1.267248 4.25604-4.447323 4.315816-4.805978H5.332005Z'/> +<path id='g2-103' d='M4.040847-1.518306C3.993026-1.327024 3.969116-1.279203 3.813699-1.099875C3.323537-.466252 2.82142-.239103 2.450809-.239103C2.056289-.239103 1.685679-.549938 1.685679-1.374844C1.685679-2.008468 2.044334-3.347447 2.307347-3.88543C2.654047-4.554919 3.19203-5.033126 3.694147-5.033126C4.483188-5.033126 4.638605-4.052802 4.638605-3.981071L4.60274-3.813699L4.040847-1.518306ZM4.782067-4.483188C4.62665-4.829888 4.291905-5.272229 3.694147-5.272229C2.391034-5.272229 .908593-3.634371 .908593-1.853051C.908593-.609714 1.661768 0 2.426899 0C3.060523 0 3.622416-.502117 3.837609-.74122L3.574595 .334745C3.407223 .992279 3.335492 1.291158 2.905106 1.709589C2.414944 2.199751 1.960648 2.199751 1.697634 2.199751C1.338979 2.199751 1.0401 2.175841 .74122 2.080199C1.123786 1.972603 1.219427 1.637858 1.219427 1.506351C1.219427 1.315068 1.075965 1.123786 .812951 1.123786C.526027 1.123786 .215193 1.362889 .215193 1.75741C.215193 2.247572 .705355 2.438854 1.721544 2.438854C3.263761 2.438854 4.064757 1.446575 4.220174 .800996L5.547198-4.554919C5.583064-4.698381 5.583064-4.722291 5.583064-4.746202C5.583064-4.913574 5.451557-5.045081 5.272229-5.045081C4.985305-5.045081 4.817933-4.805978 4.782067-4.483188Z'/> +<path id='g2-104' d='M3.359402-7.998007C3.371357-8.045828 3.395268-8.117559 3.395268-8.177335C3.395268-8.296887 3.275716-8.296887 3.251806-8.296887C3.239851-8.296887 2.654047-8.249066 2.594271-8.237111C2.391034-8.225156 2.211706-8.201245 1.996513-8.18929C1.697634-8.16538 1.613948-8.153425 1.613948-7.938232C1.613948-7.81868 1.709589-7.81868 1.876961-7.81868C2.462765-7.81868 2.47472-7.711083 2.47472-7.591532C2.47472-7.519801 2.450809-7.424159 2.438854-7.388294L.705355-.466252C.657534-.286924 .657534-.263014 .657534-.191283C.657534 .071731 .860772 .119552 .980324 .119552C1.183562 .119552 1.338979-.035866 1.398755-.167372L1.936737-2.331258C1.996513-2.594271 2.068244-2.84533 2.12802-3.108344C2.259527-3.610461 2.259527-3.622416 2.486675-3.969116S3.251806-5.033126 4.172354-5.033126C4.65056-5.033126 4.817933-4.674471 4.817933-4.196264C4.817933-3.526775 4.351681-2.223661 4.088667-1.506351C3.981071-1.219427 3.921295-1.06401 3.921295-.848817C3.921295-.310834 4.291905 .119552 4.865753 .119552C5.977584 .119552 6.396015-1.637858 6.396015-1.709589C6.396015-1.769365 6.348194-1.817186 6.276463-1.817186C6.168867-1.817186 6.156912-1.78132 6.097136-1.578082C5.822167-.621669 5.379826-.119552 4.901619-.119552C4.782067-.119552 4.590785-.131507 4.590785-.514072C4.590785-.824907 4.734247-1.207472 4.782067-1.338979C4.99726-1.912827 5.535243-3.323537 5.535243-4.016936C5.535243-4.734247 5.116812-5.272229 4.208219-5.272229C3.526775-5.272229 2.929016-4.94944 2.438854-4.327771L3.359402-7.998007Z'/> +<path id='g2-105' d='M3.383313-1.709589C3.383313-1.769365 3.335492-1.817186 3.263761-1.817186C3.156164-1.817186 3.144209-1.78132 3.084433-1.578082C2.773599-.490162 2.283437-.119552 1.888917-.119552C1.745455-.119552 1.578082-.155417 1.578082-.514072C1.578082-.836862 1.721544-1.195517 1.853051-1.554172L2.689913-3.777833C2.725778-3.873474 2.809465-4.088667 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.021171 2.139975-4.638605C2.139975-4.327771 1.984558-3.93325 1.888917-3.670237L1.052055-1.446575C.980324-1.255293 .908593-1.06401 .908593-.848817C.908593-.310834 1.279203 .119552 1.853051 .119552C2.952927 .119552 3.383313-1.625903 3.383313-1.709589ZM3.287671-7.460025C3.287671-7.639352 3.144209-7.854545 2.881196-7.854545C2.606227-7.854545 2.295392-7.591532 2.295392-7.280697C2.295392-6.981818 2.546451-6.886177 2.689913-6.886177C3.012702-6.886177 3.287671-7.197011 3.287671-7.460025Z'/> +<path id='g2-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +<path id='g2-111' d='M5.451557-3.287671C5.451557-4.423412 4.710336-5.272229 3.622416-5.272229C2.044334-5.272229 .490162-3.550685 .490162-1.865006C.490162-.729265 1.231382 .119552 2.319303 .119552C3.90934 .119552 5.451557-1.601993 5.451557-3.287671ZM2.331258-.119552C1.733499-.119552 1.291158-.597758 1.291158-1.43462C1.291158-1.984558 1.578082-3.203985 1.912827-3.801743C2.450809-4.722291 3.120299-5.033126 3.610461-5.033126C4.196264-5.033126 4.65056-4.554919 4.65056-3.718057C4.65056-3.239851 4.399502-1.960648 3.945205-1.231382C3.455044-.430386 2.797509-.119552 2.331258-.119552Z'/> +<path id='g2-112' d='M.514072 1.518306C.430386 1.876961 .382565 1.972603-.107597 1.972603C-.251059 1.972603-.37061 1.972603-.37061 2.199751C-.37061 2.223661-.358655 2.319303-.227148 2.319303C-.071731 2.319303 .095641 2.295392 .251059 2.295392H.765131C1.016189 2.295392 1.625903 2.319303 1.876961 2.319303C1.948692 2.319303 2.092154 2.319303 2.092154 2.10411C2.092154 1.972603 2.008468 1.972603 1.80523 1.972603C1.255293 1.972603 1.219427 1.888917 1.219427 1.793275C1.219427 1.649813 1.75741-.406476 1.829141-.681445C1.960648-.3467 2.283437 .119552 2.905106 .119552C4.25604 .119552 5.71457-1.637858 5.71457-3.395268C5.71457-4.495143 5.092902-5.272229 4.196264-5.272229C3.431133-5.272229 2.785554-4.531009 2.654047-4.363636C2.558406-4.961395 2.092154-5.272229 1.613948-5.272229C1.267248-5.272229 .992279-5.104857 .765131-4.65056C.549938-4.220174 .382565-3.490909 .382565-3.443088S.430386-3.335492 .514072-3.335492C.609714-3.335492 .621669-3.347447 .6934-3.622416C.872727-4.327771 1.099875-5.033126 1.578082-5.033126C1.853051-5.033126 1.948692-4.841843 1.948692-4.483188C1.948692-4.196264 1.912827-4.076712 1.865006-3.861519L.514072 1.518306ZM2.582316-3.730012C2.666002-4.064757 3.000747-4.411457 3.19203-4.578829C3.323537-4.698381 3.718057-5.033126 4.172354-5.033126C4.698381-5.033126 4.937484-4.507098 4.937484-3.88543C4.937484-3.311582 4.60274-1.960648 4.303861-1.338979C4.004981-.6934 3.455044-.119552 2.905106-.119552C2.092154-.119552 1.960648-1.147696 1.960648-1.195517C1.960648-1.231382 1.984558-1.327024 1.996513-1.3868L2.582316-3.730012Z'/> +<path id='g2-114' d='M4.65056-4.889664C4.27995-4.817933 4.088667-4.554919 4.088667-4.291905C4.088667-4.004981 4.315816-3.90934 4.483188-3.90934C4.817933-3.90934 5.092902-4.196264 5.092902-4.554919C5.092902-4.937484 4.722291-5.272229 4.124533-5.272229C3.646326-5.272229 3.096389-5.057036 2.594271-4.327771C2.510585-4.961395 2.032379-5.272229 1.554172-5.272229C1.08792-5.272229 .848817-4.913574 .705355-4.65056C.502117-4.220174 .32279-3.502864 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.812951-4.339726 1.0401-5.033126 1.518306-5.033126C1.80523-5.033126 1.888917-4.829888 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.338979 .119552 1.566127 .035866 1.637858-.203238C1.673724-.298879 2.116065-2.10411 2.187796-2.379078C2.247572-2.642092 2.319303-2.893151 2.379078-3.156164C2.426899-3.323537 2.47472-3.514819 2.510585-3.670237C2.546451-3.777833 2.86924-4.363636 3.16812-4.62665C3.311582-4.758157 3.622416-5.033126 4.112578-5.033126C4.303861-5.033126 4.495143-4.99726 4.65056-4.889664Z'/> +<path id='g2-115' d='M2.725778-2.391034C2.929016-2.355168 3.251806-2.283437 3.323537-2.271482C3.478954-2.223661 4.016936-2.032379 4.016936-1.458531C4.016936-1.08792 3.682192-.119552 2.295392-.119552C2.044334-.119552 1.147696-.155417 .908593-.812951C1.3868-.753176 1.625903-1.123786 1.625903-1.3868C1.625903-1.637858 1.458531-1.769365 1.219427-1.769365C.956413-1.769365 .609714-1.566127 .609714-1.028144C.609714-.32279 1.327024 .119552 2.283437 .119552C4.100623 .119552 4.638605-1.219427 4.638605-1.841096C4.638605-2.020423 4.638605-2.355168 4.25604-2.737733C3.957161-3.024658 3.670237-3.084433 3.024658-3.21594C2.701868-3.287671 2.187796-3.395268 2.187796-3.93325C2.187796-4.172354 2.402989-5.033126 3.53873-5.033126C4.040847-5.033126 4.531009-4.841843 4.65056-4.411457C4.124533-4.411457 4.100623-3.957161 4.100623-3.945205C4.100623-3.694147 4.327771-3.622416 4.435367-3.622416C4.60274-3.622416 4.937484-3.753923 4.937484-4.25604S4.483188-5.272229 3.550685-5.272229C1.984558-5.272229 1.566127-4.040847 1.566127-3.550685C1.566127-2.642092 2.450809-2.450809 2.725778-2.391034Z'/> +<path id='g2-116' d='M2.402989-4.805978H3.502864C3.730012-4.805978 3.849564-4.805978 3.849564-5.021171C3.849564-5.152677 3.777833-5.152677 3.53873-5.152677H2.486675L2.929016-6.898132C2.976837-7.065504 2.976837-7.089415 2.976837-7.173101C2.976837-7.364384 2.82142-7.47198 2.666002-7.47198C2.570361-7.47198 2.295392-7.436115 2.199751-7.053549L1.733499-5.152677H.609714C.37061-5.152677 .263014-5.152677 .263014-4.925529C.263014-4.805978 .3467-4.805978 .573848-4.805978H1.637858L.848817-1.649813C.753176-1.231382 .71731-1.111831 .71731-.956413C.71731-.394521 1.111831 .119552 1.78132 .119552C2.988792 .119552 3.634371-1.625903 3.634371-1.709589C3.634371-1.78132 3.58655-1.817186 3.514819-1.817186C3.490909-1.817186 3.443088-1.817186 3.419178-1.769365C3.407223-1.75741 3.395268-1.745455 3.311582-1.554172C3.060523-.956413 2.510585-.119552 1.817186-.119552C1.458531-.119552 1.43462-.418431 1.43462-.681445C1.43462-.6934 1.43462-.920548 1.470486-1.06401L2.402989-4.805978Z'/> +<path id='g2-119' d='M4.112578-.729265C4.375592-.011955 5.116812 .119552 5.571108 .119552C6.479701 .119552 7.017684-.669489 7.352428-1.494396C7.627397-2.187796 8.069738-3.765878 8.069738-4.471233C8.069738-5.200498 7.699128-5.272229 7.591532-5.272229C7.304608-5.272229 7.041594-4.985305 7.041594-4.746202C7.041594-4.60274 7.12528-4.519054 7.185056-4.471233C7.292653-4.363636 7.603487-4.040847 7.603487-3.419178C7.603487-3.012702 7.268742-1.888917 7.017684-1.327024C6.682939-.597758 6.240598-.119552 5.618929-.119552C4.94944-.119552 4.734247-.621669 4.734247-1.171606C4.734247-1.518306 4.841843-1.936737 4.889664-2.139975L5.391781-4.148443C5.451557-4.387547 5.559153-4.805978 5.559153-4.853798C5.559153-5.033126 5.415691-5.152677 5.236364-5.152677C4.889664-5.152677 4.805978-4.853798 4.734247-4.566874C4.614695-4.100623 4.112578-2.080199 4.064757-1.817186C4.016936-1.613948 4.016936-1.482441 4.016936-1.219427C4.016936-.932503 3.634371-.478207 3.622416-.454296C3.490909-.32279 3.299626-.119552 2.929016-.119552C1.984558-.119552 1.984558-1.016189 1.984558-1.219427C1.984558-1.601993 2.068244-2.12802 2.606227-3.550685C2.749689-3.921295 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.920548-4.60274 1.3868-5.033126 1.829141-5.033126C1.948692-5.033126 2.139975-5.021171 2.139975-4.638605C2.139975-4.578829 2.139975-4.327771 1.936737-3.801743C1.374844-2.307347 1.255293-1.817186 1.255293-1.362889C1.255293-.107597 2.283437 .119552 2.893151 .119552C3.096389 .119552 3.634371 .119552 4.112578-.729265Z'/> +<path id='g2-120' d='M5.66675-4.877709C5.284184-4.805978 5.140722-4.519054 5.140722-4.291905C5.140722-4.004981 5.36787-3.90934 5.535243-3.90934C5.893898-3.90934 6.144956-4.220174 6.144956-4.542964C6.144956-5.045081 5.571108-5.272229 5.068991-5.272229C4.339726-5.272229 3.93325-4.554919 3.825654-4.327771C3.550685-5.224408 2.809465-5.272229 2.594271-5.272229C1.374844-5.272229 .729265-3.706102 .729265-3.443088C.729265-3.395268 .777086-3.335492 .860772-3.335492C.956413-3.335492 .980324-3.407223 1.004234-3.455044C1.41071-4.782067 2.211706-5.033126 2.558406-5.033126C3.096389-5.033126 3.203985-4.531009 3.203985-4.244085C3.203985-3.981071 3.132254-3.706102 2.988792-3.132254L2.582316-1.494396C2.402989-.777086 2.056289-.119552 1.422665-.119552C1.362889-.119552 1.06401-.119552 .812951-.274969C1.243337-.358655 1.338979-.71731 1.338979-.860772C1.338979-1.099875 1.159651-1.243337 .932503-1.243337C.645579-1.243337 .334745-.992279 .334745-.609714C.334745-.107597 .896638 .119552 1.41071 .119552C1.984558 .119552 2.391034-.334745 2.642092-.824907C2.833375-.119552 3.431133 .119552 3.873474 .119552C5.092902 .119552 5.738481-1.446575 5.738481-1.709589C5.738481-1.769365 5.69066-1.817186 5.618929-1.817186C5.511333-1.817186 5.499377-1.75741 5.463512-1.661768C5.140722-.609714 4.447323-.119552 3.90934-.119552C3.490909-.119552 3.263761-.430386 3.263761-.920548C3.263761-1.183562 3.311582-1.374844 3.502864-2.163885L3.921295-3.789788C4.100623-4.507098 4.507098-5.033126 5.057036-5.033126C5.080946-5.033126 5.415691-5.033126 5.66675-4.877709Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -61.132812)'> +<use x='56.413267' y='71.232862' xlink:href='#g2-119'/> +<use x='65.102807' y='66.296676' xlink:href='#g3-40'/> +<use x='68.396061' y='66.296676' xlink:href='#g1-105'/> +<use x='71.2792' y='66.296676' xlink:href='#g3-41'/> +<use x='78.391415' y='71.232862' xlink:href='#g4-61'/> +<use x='90.816896' y='54.375924' xlink:href='#g0-26'/> +<use x='102.709782' y='63.910285' xlink:href='#g4-49'/> +<use x='108.562773' y='63.910285' xlink:href='#g2-59'/> +<use x='124.703569' y='63.910285' xlink:href='#g2-105'/> +<use x='128.697001' y='63.910285' xlink:href='#g2-102'/> +<use x='139.645431' y='63.910285' xlink:href='#g2-120'/> +<use x='146.297518' y='63.910285' xlink:href='#g4-40'/> +<use x='150.849844' y='63.910285' xlink:href='#g2-105'/> +<use x='154.843276' y='63.910285' xlink:href='#g4-41'/> +<use x='163.297596' y='63.910285' xlink:href='#g2-105'/> +<use x='167.291028' y='63.910285' xlink:href='#g2-115'/> +<use x='176.707027' y='63.910285' xlink:href='#g2-110'/> +<use x='183.694633' y='63.910285' xlink:href='#g2-111'/> +<use x='189.32207' y='63.910285' xlink:href='#g2-116'/> +<use x='197.451224' y='63.910285' xlink:href='#g2-112'/> +<use x='203.326367' y='63.910285' xlink:href='#g2-111'/> +<use x='208.953804' y='63.910285' xlink:href='#g2-114'/> +<use x='214.554278' y='63.910285' xlink:href='#g2-110'/> +<use x='221.541883' y='63.910285' xlink:href='#g2-111'/> +<use x='227.169321' y='63.910285' xlink:href='#g2-103'/> +<use x='233.203577' y='63.910285' xlink:href='#g2-114'/> +<use x='238.804051' y='63.910285' xlink:href='#g2-97'/> +<use x='244.948995' y='63.910285' xlink:href='#g2-112'/> +<use x='250.824138' y='63.910285' xlink:href='#g2-104'/> +<use x='257.562693' y='63.910285' xlink:href='#g2-105'/> +<use x='261.556125' y='63.910285' xlink:href='#g2-99'/> +<use x='99.783287' y='78.356097' xlink:href='#g4-49'/> +<use x='105.636277' y='78.356097' xlink:href='#g4-48'/> +<use x='111.489268' y='78.356097' xlink:href='#g2-59'/> +<use x='135.075669' y='78.356097' xlink:href='#g2-105'/> +<use x='139.069101' y='78.356097' xlink:href='#g2-102'/> +<use x='150.017531' y='78.356097' xlink:href='#g2-120'/> +<use x='156.669619' y='78.356097' xlink:href='#g4-40'/> +<use x='161.221944' y='78.356097' xlink:href='#g2-105'/> +<use x='165.215377' y='78.356097' xlink:href='#g4-41'/> +<use x='173.669696' y='78.356097' xlink:href='#g2-105'/> +<use x='177.663128' y='78.356097' xlink:href='#g2-115'/> +<use x='187.079127' y='78.356097' xlink:href='#g2-112'/> +<use x='192.95427' y='78.356097' xlink:href='#g2-111'/> +<use x='198.581708' y='78.356097' xlink:href='#g2-114'/> +<use x='204.182181' y='78.356097' xlink:href='#g2-110'/> +<use x='211.169787' y='78.356097' xlink:href='#g2-111'/> +<use x='216.797225' y='78.356097' xlink:href='#g2-103'/> +<use x='222.831481' y='78.356097' xlink:href='#g2-114'/> +<use x='228.431954' y='78.356097' xlink:href='#g2-97'/> +<use x='234.576899' y='78.356097' xlink:href='#g2-112'/> +<use x='240.452042' y='78.356097' xlink:href='#g2-104'/> +<use x='247.190597' y='78.356097' xlink:href='#g2-105'/> +<use x='251.184029' y='78.356097' xlink:href='#g2-99'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/metric-clf-error.svg b/_resources/metric-clf-error.svg new file mode 100644 index 0000000..5ca6113 --- /dev/null +++ b/_resources/metric-clf-error.svg @@ -0,0 +1,58 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='130.17237pt' height='36.972014pt' viewBox='-.244117 -.232683 130.17237 36.972014'> +<defs> +<path id='g5-40' d='M2.654047 1.992528C2.717808 1.992528 2.81345 1.992528 2.81345 1.896887C2.81345 1.865006 2.805479 1.857036 2.701868 1.753425C1.609963 .72528 1.338979-.757161 1.338979-1.992528C1.338979-4.28792 2.287422-5.363885 2.693898-5.730511C2.805479-5.834122 2.81345-5.842092 2.81345-5.881943S2.781569-5.977584 2.701868-5.977584C2.574346-5.977584 2.175841-5.571108 2.11208-5.499377C1.044085-4.383562 .820922-2.948941 .820922-1.992528C.820922-.207223 1.570112 1.227397 2.654047 1.992528Z'/> +<path id='g5-41' d='M2.462765-1.992528C2.462765-2.749689 2.335243-3.658281 1.841096-4.598755C1.45056-5.332005 .72528-5.977584 .581818-5.977584C.502117-5.977584 .478207-5.921793 .478207-5.881943C.478207-5.850062 .478207-5.834122 .573848-5.738481C1.689664-4.678456 1.944707-3.219925 1.944707-1.992528C1.944707 .294894 .996264 1.378829 .589788 1.745455C.486177 1.849066 .478207 1.857036 .478207 1.896887S.502117 1.992528 .581818 1.992528C.70934 1.992528 1.107846 1.586052 1.171606 1.514321C2.239601 .398506 2.462765-1.036115 2.462765-1.992528Z'/> +<path id='g5-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g5-61' d='M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z'/> +<path id='g0-88' d='M15.135243 16.737235L16.581818 12.911582H16.282939C15.816687 14.154919 14.54944 14.96787 13.174595 15.326526C12.923537 15.386301 11.75193 15.697136 9.456538 15.697136H2.247572L8.332752 8.5599C8.416438 8.464259 8.440349 8.428394 8.440349 8.368618C8.440349 8.344707 8.440349 8.308842 8.356663 8.18929L2.785554 .573848H9.336986C10.938979 .573848 12.026899 .74122 12.134496 .765131C12.780075 .860772 13.820174 1.06401 14.764633 1.661768C15.063512 1.853051 15.876463 2.391034 16.282939 3.359402H16.581818L15.135243 0H1.004234C.729265 0 .71731 .011955 .681445 .083686C.669489 .119552 .669489 .3467 .669489 .478207L6.993773 9.133748L.800996 16.390535C.681445 16.533998 .681445 16.593773 .681445 16.605729C.681445 16.737235 .789041 16.737235 1.004234 16.737235H15.135243Z'/> +<path id='g2-100' d='M3.616438-3.969116C3.622416-3.993026 3.634371-4.028892 3.634371-4.05878C3.634371-4.154421 3.514819-4.148443 3.443088-4.142466L2.773599-4.088667C2.67198-4.08269 2.594271-4.076712 2.594271-3.939228C2.594271-3.843587 2.67198-3.843587 2.761644-3.843587C2.940971-3.843587 2.982814-3.831631 3.060523-3.801743C3.054545-3.71208 3.054545-3.700125 3.036613-3.622416C2.911083-3.108344 2.815442-2.707846 2.69589-2.27746C2.612204-2.414944 2.408966-2.636115 2.038356-2.636115C1.273225-2.636115 .448319-1.835118 .448319-.956413C.448319-.310834 .902615 .059776 1.41071 .059776C1.811208 .059776 2.15193-.215193 2.30137-.364633C2.414944 .011955 2.815442 .059776 2.946949 .059776C3.162142 .059776 3.317559-.059776 3.431133-.245081C3.580573-.484184 3.664259-.830884 3.664259-.860772C3.664259-.872727 3.658281-.944458 3.550685-.944458C3.461021-.944458 3.449066-.902615 3.425156-.806974C3.329514-.442341 3.203985-.137484 2.970859-.137484C2.767621-.137484 2.749689-.352677 2.749689-.442341C2.749689-.52005 2.749689-.537983 2.779577-.645579L3.616438-3.969116ZM2.32528-.783064C2.295392-.675467 2.295392-.663512 2.211706-.573848C1.882939-.203238 1.578082-.137484 1.428643-.137484C1.189539-.137484 .956413-.298879 .956413-.723288C.956413-.968369 1.081943-1.554172 1.273225-1.894894C1.452553-2.217684 1.75741-2.438854 2.044334-2.438854C2.492653-2.438854 2.606227-1.966625 2.606227-1.924782L2.588294-1.841096L2.32528-.783064Z'/> +<path id='g2-101' d='M1.321046-1.344956C1.715567-1.344956 2.169863-1.374844 2.480697-1.476463C3.000747-1.649813 3.024658-2.014446 3.024658-2.10411C3.024658-2.397011 2.737733-2.636115 2.229639-2.636115C1.494396-2.636115 .478207-2.110087 .478207-1.08792C.478207-.454296 .89066 .059776 1.637858 .059776C2.749689 .059776 3.186052-.532005 3.186052-.603736C3.186052-.657534 3.120299-.723288 3.072478-.723288C3.04259-.723288 3.036613-.71731 2.946949-.627646C2.588294-.221171 1.97858-.137484 1.649813-.137484C1.165629-.137484 .992279-.484184 .992279-.86675C.992279-.91457 .998257-1.111831 1.058032-1.344956H1.321046ZM1.111831-1.542217C1.309091-2.247572 1.894894-2.438854 2.229639-2.438854C2.510585-2.438854 2.749689-2.313325 2.749689-2.098132C2.749689-1.542217 1.751432-1.542217 1.111831-1.542217Z'/> +<path id='g2-118' d='M3.425156-2.145953C3.425156-2.576339 3.180075-2.642092 3.102366-2.642092C2.929016-2.642092 2.767621-2.456787 2.767621-2.313325C2.767621-2.199751 2.84533-2.139975 2.881196-2.110087C3.000747-2.020423 3.108344-1.876961 3.108344-1.649813C3.108344-1.338979 2.677958-.137484 1.97858-.137484C1.458531-.137484 1.458531-.555915 1.458531-.669489C1.458531-.926526 1.542217-1.195517 1.799253-1.835118C1.847073-1.960648 1.865006-2.008468 1.865006-2.116065C1.865006-2.450809 1.560149-2.636115 1.267248-2.636115C.657534-2.636115 .364633-1.841096 .364633-1.715567C.364633-1.685679 .388543-1.63188 .472229-1.63188S.573848-1.667746 .591781-1.721544C.759153-2.289415 1.069988-2.438854 1.243337-2.438854C1.362889-2.438854 1.404732-2.361146 1.404732-2.223661C1.404732-2.098132 1.327024-1.894894 1.26127-1.733499C.986301-1.046077 .986301-.86675 .986301-.74122C.986301-.101619 1.530262 .059776 1.95467 .059776C2.952927 .059776 3.425156-1.554172 3.425156-2.145953Z'/> +<path id='g3-100' d='M4.28792-5.292154C4.29589-5.308095 4.319801-5.411706 4.319801-5.419676C4.319801-5.459527 4.28792-5.531258 4.192279-5.531258C4.160399-5.531258 3.913325-5.507347 3.730012-5.491407L3.283686-5.459527C3.108344-5.443587 3.028643-5.435616 3.028643-5.292154C3.028643-5.180573 3.140224-5.180573 3.235866-5.180573C3.618431-5.180573 3.618431-5.132752 3.618431-5.061021C3.618431-5.0132 3.55467-4.750187 3.514819-4.590785L3.124284-3.036613C3.052553-3.172105 2.82142-3.514819 2.335243-3.514819C1.3868-3.514819 .342715-2.406974 .342715-1.227397C.342715-.398506 .876712 .079701 1.490411 .079701C2.000498 .079701 2.438854-.326775 2.582316-.486177C2.725778 .063761 3.267746 .079701 3.363387 .079701C3.730012 .079701 3.913325-.223163 3.977086-.358655C4.136488-.645579 4.24807-1.107846 4.24807-1.139726C4.24807-1.187547 4.216189-1.243337 4.120548-1.243337S4.008966-1.195517 3.961146-.996264C3.849564-.557908 3.698132-.143462 3.387298-.143462C3.203985-.143462 3.132254-.294894 3.132254-.518057C3.132254-.669489 3.156164-.757161 3.180075-.860772L4.28792-5.292154ZM2.582316-.860772C2.183811-.310834 1.769365-.143462 1.514321-.143462C1.147696-.143462 .964384-.478207 .964384-.892653C.964384-1.267248 1.179577-2.12005 1.354919-2.470735C1.586052-2.956912 1.976588-3.291656 2.343213-3.291656C2.86127-3.291656 3.012702-2.709838 3.012702-2.614197C3.012702-2.582316 2.81345-1.801245 2.765629-1.594022C2.662017-1.219427 2.662017-1.203487 2.582316-.860772Z'/> +<path id='g3-101' d='M1.601993-1.817186C1.777335-1.817186 2.375093-1.825156 2.797509-1.976588C3.490909-2.215691 3.514819-2.693898 3.514819-2.81345C3.514819-3.259776 3.092403-3.514819 2.574346-3.514819C1.673724-3.514819 .390535-2.805479 .390535-1.39477C.390535-.581818 .884682 .079701 1.761395 .079701C3.004732 .079701 3.674222-.71731 3.674222-.828892C3.674222-.900623 3.594521-.956413 3.5467-.956413S3.474969-.932503 3.435118-.884682C2.805479-.143462 1.912827-.143462 1.777335-.143462C1.195517-.143462 1.020174-.637609 1.020174-1.083935C1.020174-1.323039 1.091905-1.681694 1.123786-1.817186H1.601993ZM1.187547-2.040349C1.44259-3.012702 2.16787-3.291656 2.574346-3.291656C2.893151-3.291656 3.196015-3.132254 3.196015-2.81345C3.196015-2.040349 1.888917-2.040349 1.554172-2.040349H1.187547Z'/> +<path id='g3-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g3-109' d='M1.594022-1.307098C1.617933-1.42665 1.697634-1.729514 1.721544-1.849066C1.745455-1.928767 1.793275-2.12005 1.809215-2.199751C1.825156-2.239601 2.088169-2.757659 2.438854-3.020672C2.709838-3.227895 2.972852-3.291656 3.196015-3.291656C3.490909-3.291656 3.650311-3.116314 3.650311-2.749689C3.650311-2.558406 3.602491-2.375093 3.514819-2.016438C3.459029-1.809215 3.323537-1.275218 3.275716-1.060025L3.156164-.581818C3.116314-.446326 3.060523-.207223 3.060523-.167372C3.060523 .01594 3.211955 .079701 3.315567 .079701C3.459029 .079701 3.57858-.01594 3.634371-.111582C3.658281-.159402 3.722042-.430386 3.761893-.597758L3.945205-1.307098C3.969116-1.42665 4.048817-1.729514 4.072727-1.849066C4.184309-2.279452 4.184309-2.287422 4.367621-2.550436C4.630635-2.940971 5.00523-3.291656 5.539228-3.291656C5.826152-3.291656 5.993524-3.124284 5.993524-2.749689C5.993524-2.311333 5.65878-1.39477 5.507347-1.012204C5.427646-.804981 5.403736-.749191 5.403736-.597758C5.403736-.143462 5.778331 .079701 6.121046 .079701C6.902117 .079701 7.228892-1.036115 7.228892-1.139726C7.228892-1.219427 7.165131-1.243337 7.10934-1.243337C7.013699-1.243337 6.997758-1.187547 6.973848-1.107846C6.782565-.446326 6.447821-.143462 6.144956-.143462C6.017435-.143462 5.953674-.223163 5.953674-.406476S6.017435-.765131 6.097136-.964384C6.216687-1.267248 6.567372-2.183811 6.567372-2.630137C6.567372-3.227895 6.152927-3.514819 5.579078-3.514819C5.029141-3.514819 4.574844-3.227895 4.216189-2.733748C4.152428-3.371357 3.642341-3.514819 3.227895-3.514819C2.86127-3.514819 2.375093-3.387298 1.936737-2.81345C1.880946-3.291656 1.498381-3.514819 1.123786-3.514819C.844832-3.514819 .645579-3.347447 .510087-3.076463C.318804-2.701868 .239103-2.311333 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.223661 .526027-2.430884C.621669-2.82142 .765131-3.291656 1.099875-3.291656C1.307098-3.291656 1.354919-3.092403 1.354919-2.917061C1.354919-2.773599 1.315068-2.622167 1.251308-2.359153C1.235367-2.295392 1.115816-1.825156 1.083935-1.713574L.789041-.518057C.757161-.398506 .70934-.199253 .70934-.167372C.70934 .01594 .860772 .079701 .964384 .079701C1.107846 .079701 1.227397-.01594 1.283188-.111582C1.307098-.159402 1.370859-.430386 1.41071-.597758L1.594022-1.307098Z'/> +<path id='g3-118' d='M3.961146-2.901121C3.961146-3.52279 3.602491-3.52279 3.57061-3.52279C3.379328-3.52279 3.156164-3.315567 3.156164-3.108344C3.156164-2.980822 3.219925-2.925031 3.291656-2.86127C3.474969-2.701868 3.58655-2.478705 3.58655-2.223661C3.58655-1.857036 3.060523-.143462 2.14396-.143462C1.801245-.143462 1.522291-.326775 1.522291-.828892C1.522291-1.267248 1.761395-1.896887 1.952677-2.375093C2.048319-2.630137 2.072229-2.693898 2.072229-2.83736C2.072229-3.267746 1.721544-3.514819 1.354919-3.514819C.565878-3.514819 .239103-2.391034 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.70137-3.012702 1.044085-3.291656 1.331009-3.291656C1.45056-3.291656 1.522291-3.219925 1.522291-3.028643C1.522291-2.84533 1.45056-2.662017 1.370859-2.462765C.980324-1.458531 .948443-1.195517 .948443-.948443C.948443-.079701 1.657783 .079701 2.11208 .079701C3.435118 .079701 3.961146-2.295392 3.961146-2.901121Z'/> +<path id='g4-109' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-4.124533 5.057036-3.88543 4.913574-3.323537L4.614695-2.092154C4.519054-1.733499 4.291905-.848817 4.267995-.71731C4.220174-.537983 4.148443-.227148 4.148443-.179328C4.148443-.011955 4.27995 .119552 4.459278 .119552C4.817933 .119552 4.877709-.155417 4.985305-.585803L5.702615-3.443088C5.726526-3.53873 6.348194-5.033126 7.663263-5.033126C8.141469-5.033126 8.308842-4.674471 8.308842-4.196264C8.308842-3.526775 7.84259-2.223661 7.579577-1.506351C7.47198-1.219427 7.412204-1.06401 7.412204-.848817C7.412204-.310834 7.782814 .119552 8.356663 .119552C9.468493 .119552 9.886924-1.637858 9.886924-1.709589C9.886924-1.769365 9.839103-1.817186 9.767372-1.817186C9.659776-1.817186 9.647821-1.78132 9.588045-1.578082C9.313076-.621669 8.870735-.119552 8.392528-.119552C8.272976-.119552 8.081694-.131507 8.081694-.514072C8.081694-.824907 8.225156-1.207472 8.272976-1.338979C8.488169-1.912827 9.026152-3.323537 9.026152-4.016936C9.026152-4.734247 8.607721-5.272229 7.699128-5.272229C6.898132-5.272229 6.252553-4.817933 5.774346-4.112578C5.738481-4.758157 5.34396-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.207472-5.272229 .932503-5.104857 .705355-4.65056C.490162-4.220174 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.812951-4.327771 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +<path id='g4-121' d='M3.144209 1.338979C2.82142 1.793275 2.355168 2.199751 1.769365 2.199751C1.625903 2.199751 1.052055 2.175841 .872727 1.625903C.908593 1.637858 .968369 1.637858 .992279 1.637858C1.350934 1.637858 1.590037 1.327024 1.590037 1.052055S1.362889 .681445 1.183562 .681445C.992279 .681445 .573848 .824907 .573848 1.41071C.573848 2.020423 1.08792 2.438854 1.769365 2.438854C2.964882 2.438854 4.172354 1.338979 4.507098 .011955L5.678705-4.65056C5.69066-4.710336 5.71457-4.782067 5.71457-4.853798C5.71457-5.033126 5.571108-5.152677 5.391781-5.152677C5.284184-5.152677 5.033126-5.104857 4.937484-4.746202L4.052802-1.231382C3.993026-1.016189 3.993026-.992279 3.897385-.860772C3.658281-.526027 3.263761-.119552 2.689913-.119552C2.020423-.119552 1.960648-.777086 1.960648-1.099875C1.960648-1.78132 2.283437-2.701868 2.606227-3.56264C2.737733-3.90934 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.033126 2.139975-4.638605C2.139975-4.327771 2.008468-3.981071 1.829141-3.526775C1.243337-1.960648 1.243337-1.566127 1.243337-1.279203C1.243337-.143462 2.056289 .119552 2.654047 .119552C3.000747 .119552 3.431133 .011955 3.849564-.430386L3.861519-.418431C3.682192 .286924 3.56264 .753176 3.144209 1.338979Z'/> +<path id='g1-54' d='M7.531756-8.093649C7.627397-8.261021 7.627397-8.284932 7.627397-8.320797C7.627397-8.404483 7.555666-8.5599 7.388294-8.5599C7.244832-8.5599 7.208966-8.488169 7.12528-8.320797L1.75741 2.116065C1.661768 2.283437 1.661768 2.307347 1.661768 2.343213C1.661768 2.438854 1.745455 2.582316 1.900872 2.582316C2.044334 2.582316 2.080199 2.510585 2.163885 2.343213L7.531756-8.093649Z'/> +<path id='g1-76' d='M2.15193-1.111831C2.797509-2.116065 3.000747-2.881196 3.156164-3.514819C3.574595-5.164633 4.028892-6.599253 4.770112-7.424159C4.913574-7.579577 5.009215-7.687173 5.391781-7.687173C6.216687-7.687173 6.240598-6.862267 6.240598-6.694894C6.240598-6.479701 6.180822-6.312329 6.180822-6.252553C6.180822-6.168867 6.252553-6.168867 6.264508-6.168867C6.455791-6.168867 6.77858-6.300374 7.07746-6.515567C7.292653-6.682939 7.400249-6.802491 7.400249-7.292653C7.400249-7.938232 7.065504-8.428394 6.396015-8.428394C6.01345-8.428394 4.961395-8.332752 3.789788-7.149191C2.833375-6.168867 2.271482-4.016936 2.044334-3.120299C1.829141-2.295392 1.733499-1.924782 1.374844-1.207472C1.291158-1.06401 .980324-.537983 .812951-.382565C.490162-.083686 .37061 .131507 .37061 .191283C.37061 .215193 .394521 .263014 .478207 .263014C.526027 .263014 .777086 .215193 1.08792 .011955C1.291158-.107597 1.315068-.131507 1.590037-.418431C2.187796-.406476 2.606227-.298879 3.359402-.083686C3.969116 .083686 4.578829 .263014 5.188543 .263014C6.156912 .263014 7.137235-.466252 7.519801-.992279C7.758904-1.315068 7.830635-1.613948 7.830635-1.649813C7.830635-1.733499 7.758904-1.733499 7.746949-1.733499C7.555666-1.733499 7.268742-1.601993 7.065504-1.458531C6.742715-1.255293 6.718804-1.183562 6.647073-.980324C6.587298-.789041 6.515567-.6934 6.467746-.621669C6.372105-.478207 6.360149-.478207 6.180822-.478207C5.606974-.478207 5.009215-.657534 4.220174-.872727C3.88543-.968369 3.227895-1.159651 2.630137-1.159651C2.47472-1.159651 2.307347-1.147696 2.15193-1.111831Z'/> +<path id='g1-102' d='M3.383313-7.376339C3.383313-7.854545 3.694147-8.619676 4.99726-8.703362C5.057036-8.715318 5.104857-8.763138 5.104857-8.834869C5.104857-8.966376 5.009215-8.966376 4.877709-8.966376C3.682192-8.966376 2.594271-8.356663 2.582316-7.47198V-4.746202C2.582316-4.27995 2.582316-3.897385 2.10411-3.502864C1.685679-3.156164 1.231382-3.132254 .968369-3.120299C.908593-3.108344 .860772-3.060523 .860772-2.988792C.860772-2.86924 .932503-2.86924 1.052055-2.857285C1.841096-2.809465 2.414944-2.379078 2.546451-1.793275C2.582316-1.661768 2.582316-1.637858 2.582316-1.207472V1.159651C2.582316 1.661768 2.582316 2.044334 3.156164 2.49863C3.622416 2.857285 4.411457 2.988792 4.877709 2.988792C5.009215 2.988792 5.104857 2.988792 5.104857 2.857285C5.104857 2.737733 5.033126 2.737733 4.913574 2.725778C4.160399 2.677958 3.574595 2.295392 3.419178 1.685679C3.383313 1.578082 3.383313 1.554172 3.383313 1.123786V-1.3868C3.383313-1.936737 3.287671-2.139975 2.905106-2.52254C2.654047-2.773599 2.307347-2.893151 1.972603-2.988792C2.952927-3.263761 3.383313-3.813699 3.383313-4.507098V-7.376339Z'/> +<path id='g1-103' d='M2.582316 1.398755C2.582316 1.876961 2.271482 2.642092 .968369 2.725778C.908593 2.737733 .860772 2.785554 .860772 2.857285C.860772 2.988792 .992279 2.988792 1.099875 2.988792C2.259527 2.988792 3.371357 2.402989 3.383313 1.494396V-1.231382C3.383313-1.697634 3.383313-2.080199 3.861519-2.47472C4.27995-2.82142 4.734247-2.84533 4.99726-2.857285C5.057036-2.86924 5.104857-2.917061 5.104857-2.988792C5.104857-3.108344 5.033126-3.108344 4.913574-3.120299C4.124533-3.16812 3.550685-3.598506 3.419178-4.184309C3.383313-4.315816 3.383313-4.339726 3.383313-4.770112V-7.137235C3.383313-7.639352 3.383313-8.021918 2.809465-8.476214C2.331258-8.846824 1.506351-8.966376 1.099875-8.966376C.992279-8.966376 .860772-8.966376 .860772-8.834869C.860772-8.715318 .932503-8.715318 1.052055-8.703362C1.80523-8.655542 2.391034-8.272976 2.546451-7.663263C2.582316-7.555666 2.582316-7.531756 2.582316-7.10137V-4.590785C2.582316-4.040847 2.677958-3.837609 3.060523-3.455044C3.311582-3.203985 3.658281-3.084433 3.993026-2.988792C3.012702-2.713823 2.582316-2.163885 2.582316-1.470486V1.398755Z'/> +<path id='g6-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +<path id='g6-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g6-94' d='M2.929016-8.296887L1.362889-6.670984L1.554172-6.491656L2.917061-7.723039L4.291905-6.491656L4.483188-6.670984L2.929016-8.296887Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -65.34204 -62.281577)'> +<use x='66.398558' y='65.624772' xlink:href='#g6-49'/> +<rect x='57.608781' y='70.484645' height='.478187' width='23.432544'/> +<use x='57.608781' y='81.913193' xlink:href='#g4-109'/> +<use x='67.848048' y='83.706456' xlink:href='#g3-100'/> +<use x='72.205365' y='83.706456' xlink:href='#g3-101'/> +<use x='76.133749' y='83.706456' xlink:href='#g3-118'/> +<use x='84.229337' y='58.425345' xlink:href='#g3-109'/> +<use x='91.719863' y='59.83119' xlink:href='#g2-100'/> +<use x='95.560318' y='59.83119' xlink:href='#g2-101'/> +<use x='99.081142' y='59.83119' xlink:href='#g2-118'/> +<use x='85.245196' y='62.355066' xlink:href='#g0-88'/> +<use x='87.027589' y='87.549421' xlink:href='#g3-105'/> +<use x='89.910729' y='87.549421' xlink:href='#g5-61'/> +<use x='96.497236' y='87.549421' xlink:href='#g5-49'/> +<use x='105.522169' y='73.712531' xlink:href='#g1-76'/> +<use x='113.767946' y='73.712531' xlink:href='#g1-102'/> +<use x='120.537718' y='73.712531' xlink:href='#g6-94'/> +<use x='119.745553' y='73.712531' xlink:href='#g4-121'/> +<use x='125.882205' y='68.776345' xlink:href='#g5-40'/> +<use x='129.175458' y='68.776345' xlink:href='#g3-105'/> +<use x='132.058598' y='68.776345' xlink:href='#g5-41'/> +<use x='139.170801' y='73.712531' xlink:href='#g1-54'/> +<use x='139.170801' y='73.712531' xlink:href='#g6-61'/> +<use x='151.596282' y='73.712531' xlink:href='#g4-121'/> +<use x='157.732934' y='68.776345' xlink:href='#g5-40'/> +<use x='161.026187' y='68.776345' xlink:href='#g3-105'/> +<use x='163.909327' y='68.776345' xlink:href='#g5-41'/> +<use x='167.700712' y='73.712531' xlink:href='#g1-103'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/momentum-algo.png b/_resources/momentum-algo.png new file mode 100644 index 0000000..40281c6 Binary files /dev/null and b/_resources/momentum-algo.png differ diff --git a/_resources/mt-conditional-lm.png b/_resources/mt-conditional-lm.png new file mode 100644 index 0000000..54d2b9c Binary files /dev/null and b/_resources/mt-conditional-lm.png differ diff --git a/_resources/multi-crop.png b/_resources/multi-crop.png new file mode 100644 index 0000000..6727a31 Binary files /dev/null and b/_resources/multi-crop.png differ diff --git a/_resources/multi-task-learning.png b/_resources/multi-task-learning.png new file mode 100644 index 0000000..c7c39f5 Binary files /dev/null and b/_resources/multi-task-learning.png differ diff --git a/_resources/n_x_64_64_3_12288.svg b/_resources/n_x_64_64_3_12288.svg new file mode 100644 index 0000000..85ca9de --- /dev/null +++ b/_resources/n_x_64_64_3_12288.svg @@ -0,0 +1,34 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='148.031958pt' height='11.235254pt' viewBox='-.239051 -.244435 148.031958 11.235254'> +<defs> +<path id='g0-2' d='M4.65056-3.323537L2.259527-5.702615C2.116065-5.846077 2.092154-5.869988 1.996513-5.869988C1.876961-5.869988 1.75741-5.762391 1.75741-5.630884C1.75741-5.547198 1.78132-5.523288 1.912827-5.391781L4.303861-2.988792L1.912827-.585803C1.78132-.454296 1.75741-.430386 1.75741-.3467C1.75741-.215193 1.876961-.107597 1.996513-.107597C2.092154-.107597 2.116065-.131507 2.259527-.274969L4.638605-2.654047L7.113325-.179328C7.137235-.167372 7.220922-.107597 7.292653-.107597C7.436115-.107597 7.531756-.215193 7.531756-.3467C7.531756-.37061 7.531756-.418431 7.49589-.478207C7.483935-.502117 5.583064-2.379078 4.985305-2.988792L7.173101-5.176588C7.232877-5.248319 7.412204-5.403736 7.47198-5.475467C7.483935-5.499377 7.531756-5.547198 7.531756-5.630884C7.531756-5.762391 7.436115-5.869988 7.292653-5.869988C7.197011-5.869988 7.149191-5.822167 7.017684-5.69066L4.65056-3.323537Z'/> +<path id='g3-49' d='M3.443088-7.663263C3.443088-7.938232 3.443088-7.950187 3.203985-7.950187C2.917061-7.627397 2.319303-7.185056 1.08792-7.185056V-6.838356C1.362889-6.838356 1.960648-6.838356 2.618182-7.149191V-.920548C2.618182-.490162 2.582316-.3467 1.530262-.3467H1.159651V0C1.482441-.02391 2.642092-.02391 3.036613-.02391S4.578829-.02391 4.901619 0V-.3467H4.531009C3.478954-.3467 3.443088-.490162 3.443088-.920548V-7.663263Z'/> +<path id='g3-50' d='M5.260274-2.008468H4.99726C4.961395-1.80523 4.865753-1.147696 4.746202-.956413C4.662516-.848817 3.981071-.848817 3.622416-.848817H1.41071C1.733499-1.123786 2.462765-1.888917 2.773599-2.175841C4.590785-3.849564 5.260274-4.471233 5.260274-5.654795C5.260274-7.029639 4.172354-7.950187 2.785554-7.950187S.585803-6.766625 .585803-5.738481C.585803-5.128767 1.111831-5.128767 1.147696-5.128767C1.398755-5.128767 1.709589-5.308095 1.709589-5.69066C1.709589-6.025405 1.482441-6.252553 1.147696-6.252553C1.0401-6.252553 1.016189-6.252553 .980324-6.240598C1.207472-7.053549 1.853051-7.603487 2.630137-7.603487C3.646326-7.603487 4.267995-6.75467 4.267995-5.654795C4.267995-4.638605 3.682192-3.753923 3.000747-2.988792L.585803-.286924V0H4.94944L5.260274-2.008468Z'/> +<path id='g3-51' d='M2.199751-4.291905C1.996513-4.27995 1.948692-4.267995 1.948692-4.160399C1.948692-4.040847 2.008468-4.040847 2.223661-4.040847H2.773599C3.789788-4.040847 4.244085-3.203985 4.244085-2.056289C4.244085-.490162 3.431133-.071731 2.84533-.071731C2.271482-.071731 1.291158-.3467 .944458-1.135741C1.327024-1.075965 1.673724-1.291158 1.673724-1.721544C1.673724-2.068244 1.422665-2.307347 1.08792-2.307347C.800996-2.307347 .490162-2.139975 .490162-1.685679C.490162-.621669 1.554172 .251059 2.881196 .251059C4.303861 .251059 5.355915-.836862 5.355915-2.044334C5.355915-3.144209 4.471233-4.004981 3.323537-4.208219C4.363636-4.507098 5.033126-5.379826 5.033126-6.312329C5.033126-7.256787 4.052802-7.950187 2.893151-7.950187C1.697634-7.950187 .812951-7.220922 .812951-6.348194C.812951-5.869988 1.183562-5.774346 1.362889-5.774346C1.613948-5.774346 1.900872-5.953674 1.900872-6.312329C1.900872-6.694894 1.613948-6.862267 1.350934-6.862267C1.279203-6.862267 1.255293-6.862267 1.219427-6.850311C1.673724-7.663263 2.797509-7.663263 2.857285-7.663263C3.251806-7.663263 4.028892-7.483935 4.028892-6.312329C4.028892-6.085181 3.993026-5.415691 3.646326-4.901619C3.287671-4.375592 2.881196-4.339726 2.558406-4.327771L2.199751-4.291905Z'/> +<path id='g3-52' d='M4.315816-7.782814C4.315816-8.009963 4.315816-8.069738 4.148443-8.069738C4.052802-8.069738 4.016936-8.069738 3.921295-7.926276L.32279-2.343213V-1.996513H3.466999V-.908593C3.466999-.466252 3.443088-.3467 2.570361-.3467H2.331258V0C2.606227-.02391 3.550685-.02391 3.88543-.02391S5.176588-.02391 5.451557 0V-.3467H5.212453C4.351681-.3467 4.315816-.466252 4.315816-.908593V-1.996513H5.523288V-2.343213H4.315816V-7.782814ZM3.526775-6.850311V-2.343213H.621669L3.526775-6.850311Z'/> +<path id='g3-54' d='M1.470486-4.160399C1.470486-7.185056 2.940971-7.663263 3.58655-7.663263C4.016936-7.663263 4.447323-7.531756 4.674471-7.173101C4.531009-7.173101 4.076712-7.173101 4.076712-6.682939C4.076712-6.419925 4.25604-6.192777 4.566874-6.192777C4.865753-6.192777 5.068991-6.372105 5.068991-6.718804C5.068991-7.340473 4.614695-7.950187 3.574595-7.950187C2.068244-7.950187 .490162-6.40797 .490162-3.777833C.490162-.490162 1.924782 .251059 2.940971 .251059C4.244085 .251059 5.355915-.884682 5.355915-2.438854C5.355915-4.028892 4.244085-5.092902 3.048568-5.092902C1.984558-5.092902 1.590037-4.172354 1.470486-3.837609V-4.160399ZM2.940971-.071731C2.187796-.071731 1.829141-.74122 1.721544-.992279C1.613948-1.303113 1.494396-1.888917 1.494396-2.725778C1.494396-3.670237 1.924782-4.853798 3.000747-4.853798C3.658281-4.853798 4.004981-4.411457 4.184309-4.004981C4.375592-3.56264 4.375592-2.964882 4.375592-2.450809C4.375592-1.841096 4.375592-1.303113 4.148443-.848817C3.849564-.274969 3.419178-.071731 2.940971-.071731Z'/> +<path id='g3-56' d='M3.56264-4.315816C4.160399-4.638605 5.033126-5.188543 5.033126-6.192777C5.033126-7.232877 4.028892-7.950187 2.929016-7.950187C1.745455-7.950187 .812951-7.07746 .812951-5.989539C.812951-5.583064 .932503-5.176588 1.267248-4.770112C1.398755-4.614695 1.41071-4.60274 2.247572-4.016936C1.08792-3.478954 .490162-2.677958 .490162-1.80523C.490162-.537983 1.697634 .251059 2.917061 .251059C4.244085 .251059 5.355915-.729265 5.355915-1.984558C5.355915-3.203985 4.495143-3.741968 3.56264-4.315816ZM1.936737-5.391781C1.78132-5.499377 1.303113-5.810212 1.303113-6.396015C1.303113-7.173101 2.116065-7.663263 2.917061-7.663263C3.777833-7.663263 4.542964-7.041594 4.542964-6.180822C4.542964-5.451557 4.016936-4.865753 3.323537-4.483188L1.936737-5.391781ZM2.49863-3.849564L3.945205-2.905106C4.25604-2.701868 4.805978-2.331258 4.805978-1.601993C4.805978-.6934 3.88543-.071731 2.929016-.071731C1.912827-.071731 1.0401-.812951 1.0401-1.80523C1.0401-2.737733 1.721544-3.490909 2.49863-3.849564Z'/> +<path id='g3-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g1-120' d='M3.993026-3.180075C3.642341-3.092403 3.626401-2.781569 3.626401-2.749689C3.626401-2.574346 3.761893-2.454795 3.937235-2.454795S4.383562-2.590286 4.383562-2.933001C4.383562-3.387298 3.881445-3.514819 3.58655-3.514819C3.211955-3.514819 2.909091-3.251806 2.725778-2.940971C2.550436-3.363387 2.13599-3.514819 1.809215-3.514819C.940473-3.514819 .454296-2.518555 .454296-2.295392C.454296-2.223661 .510087-2.191781 .573848-2.191781C.669489-2.191781 .68543-2.231631 .70934-2.327273C.892653-2.909091 1.370859-3.291656 1.785305-3.291656C2.096139-3.291656 2.247572-3.068493 2.247572-2.781569C2.247572-2.622167 2.15193-2.255542 2.088169-2.000498C2.032379-1.769365 1.857036-1.060025 1.817186-.908593C1.705604-.478207 1.41868-.143462 1.060025-.143462C1.028144-.143462 .820922-.143462 .653549-.255044C1.020174-.342715 1.020174-.67746 1.020174-.68543C1.020174-.868742 .876712-.980324 .70137-.980324C.486177-.980324 .255044-.797011 .255044-.494147C.255044-.127522 .645579 .079701 1.052055 .079701C1.474471 .079701 1.769365-.239103 1.912827-.494147C2.088169-.103611 2.454795 .079701 2.83736 .079701C3.706102 .079701 4.184309-.916563 4.184309-1.139726C4.184309-1.219427 4.120548-1.243337 4.064757-1.243337C3.969116-1.243337 3.953176-1.187547 3.929265-1.107846C3.769863-.573848 3.315567-.143462 2.8533-.143462C2.590286-.143462 2.399004-.318804 2.399004-.653549C2.399004-.812951 2.446824-.996264 2.558406-1.44259C2.614197-1.681694 2.789539-2.383064 2.82939-2.534496C2.940971-2.948941 3.219925-3.291656 3.57858-3.291656C3.618431-3.291656 3.825654-3.291656 3.993026-3.180075Z'/> +<path id='g2-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.986043 -65.427)'> +<use x='56.413267' y='65.753425' xlink:href='#g2-110'/> +<use x='63.400873' y='67.546688' xlink:href='#g1-120'/> +<use x='71.986733' y='65.753425' xlink:href='#g3-61'/> +<use x='84.412214' y='65.753425' xlink:href='#g3-54'/> +<use x='90.265204' y='65.753425' xlink:href='#g3-52'/> +<use x='98.774858' y='65.753425' xlink:href='#g0-2'/> +<use x='110.730018' y='65.753425' xlink:href='#g3-54'/> +<use x='116.583009' y='65.753425' xlink:href='#g3-52'/> +<use x='125.092662' y='65.753425' xlink:href='#g0-2'/> +<use x='137.047823' y='65.753425' xlink:href='#g3-51'/> +<use x='146.221643' y='65.753425' xlink:href='#g3-61'/> +<use x='158.647123' y='65.753425' xlink:href='#g3-49'/> +<use x='164.500114' y='65.753425' xlink:href='#g3-50'/> +<use x='170.353104' y='65.753425' xlink:href='#g3-50'/> +<use x='176.206094' y='65.753425' xlink:href='#g3-56'/> +<use x='182.059084' y='65.753425' xlink:href='#g3-56'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/negative-sampling-p.svg b/_resources/negative-sampling-p.svg new file mode 100644 index 0000000..ba13c47 --- /dev/null +++ b/_resources/negative-sampling-p.svg @@ -0,0 +1,54 @@ +<?xml version='1.0' encoding='UTF-8'?> +<!-- Generated by CodeCogs with dvisvgm 2.9.1 --> +<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='130.814921pt' height='39.611683pt' viewBox='-.237481 -.227449 130.814921 39.611683'> +<defs> +<path id='g4-48' d='M3.897385-2.542466C3.897385-3.395268 3.809714-3.913325 3.5467-4.423412C3.196015-5.124782 2.550436-5.300125 2.11208-5.300125C1.107846-5.300125 .74122-4.550934 .629639-4.327771C.342715-3.745953 .326775-2.956912 .326775-2.542466C.326775-2.016438 .350685-1.211457 .73325-.573848C1.099875 .01594 1.689664 .167372 2.11208 .167372C2.494645 .167372 3.180075 .047821 3.57858-.74122C3.873474-1.315068 3.897385-2.024408 3.897385-2.542466ZM2.11208-.055791C1.841096-.055791 1.291158-.183313 1.123786-1.020174C1.036115-1.474471 1.036115-2.223661 1.036115-2.638107C1.036115-3.188045 1.036115-3.745953 1.123786-4.184309C1.291158-4.99726 1.912827-5.076961 2.11208-5.076961C2.383064-5.076961 2.933001-4.941469 3.092403-4.216189C3.188045-3.777833 3.188045-3.180075 3.188045-2.638107C3.188045-2.16787 3.188045-1.45056 3.092403-1.004234C2.925031-.167372 2.375093-.055791 2.11208-.055791Z'/> +<path id='g4-49' d='M2.502615-5.076961C2.502615-5.292154 2.486675-5.300125 2.271482-5.300125C1.944707-4.98132 1.522291-4.790037 .765131-4.790037V-4.527024C.980324-4.527024 1.41071-4.527024 1.872976-4.742217V-.653549C1.872976-.358655 1.849066-.263014 1.091905-.263014H.812951V0C1.139726-.02391 1.825156-.02391 2.183811-.02391S3.235866-.02391 3.56264 0V-.263014H3.283686C2.526526-.263014 2.502615-.358655 2.502615-.653549V-5.076961Z'/> +<path id='g4-61' d='M5.826152-2.654047C5.945704-2.654047 6.105106-2.654047 6.105106-2.83736S5.913823-3.020672 5.794271-3.020672H.781071C.661519-3.020672 .470237-3.020672 .470237-2.83736S.629639-2.654047 .749191-2.654047H5.826152ZM5.794271-.964384C5.913823-.964384 6.105106-.964384 6.105106-1.147696S5.945704-1.331009 5.826152-1.331009H.749191C.629639-1.331009 .470237-1.331009 .470237-1.147696S.661519-.964384 .781071-.964384H5.794271Z'/> +<path id='g0-80' d='M5.033126 6.38406L.789041 11.632379C.6934 11.75193 .681445 11.775841 .681445 11.823661C.681445 11.955168 .789041 11.955168 1.004234 11.955168H10.915068L11.943213 8.978331H11.644334C11.345455 9.874969 10.544458 10.604234 9.528269 10.950934C9.336986 11.01071 8.51208 11.297634 6.75467 11.297634H1.673724L5.822167 6.168867C5.905853 6.06127 5.929763 6.025405 5.929763 5.977584S5.917808 5.917808 5.846077 5.810212L1.960648 .478207H6.694894C8.057783 .478207 10.807472 .561893 11.644334 2.797509H11.943213L10.915068 0H1.004234C.681445 0 .669489 .011955 .669489 .382565L5.033126 6.38406Z'/> +<path id='g3-51' d='M1.75741-1.984558C2.265504-1.984558 2.606227-1.643836 2.606227-1.034122C2.606227-.376588 2.211706-.089664 1.775342-.089664C1.613948-.089664 .998257-.125529 .729265-.472229C.962391-.496139 1.058032-.651557 1.058032-.812951C1.058032-1.016189 .91457-1.153674 .71731-1.153674C.555915-1.153674 .376588-1.052055 .376588-.800996C.376588-.209215 1.034122 .125529 1.793275 .125529C2.677958 .125529 3.269738-.442341 3.269738-1.034122C3.269738-1.470486 2.934994-1.93076 2.223661-2.110087C2.69589-2.265504 3.078456-2.64807 3.078456-3.138232C3.078456-3.616438 2.516563-3.975093 1.799253-3.975093C1.099875-3.975093 .56787-3.652304 .56787-3.162142C.56787-2.905106 .765131-2.833375 .884682-2.833375C1.0401-2.833375 1.201494-2.940971 1.201494-3.150187C1.201494-3.335492 1.069988-3.443088 .908593-3.466999C1.189539-3.777833 1.733499-3.777833 1.787298-3.777833C2.092154-3.777833 2.47472-3.634371 2.47472-3.138232C2.47472-2.809465 2.289415-2.235616 1.691656-2.199751C1.58406-2.193773 1.422665-2.181818 1.368867-2.181818C1.309091-2.175841 1.243337-2.169863 1.243337-2.080199C1.243337-1.984558 1.309091-1.984558 1.41071-1.984558H1.75741Z'/> +<path id='g3-52' d='M3.371357-.980324V-1.21345H2.67198V-3.855542C2.67198-3.993026 2.67198-4.034869 2.52254-4.034869C2.414944-4.034869 2.402989-4.016936 2.349191-3.945205L.274969-1.21345V-.980324H2.133998V-.502117C2.133998-.304857 2.133998-.233126 1.63188-.233126H1.458531V0C1.578082-.005978 2.139975-.02391 2.402989-.02391S3.227895-.005978 3.347447 0V-.233126H3.174097C2.67198-.233126 2.67198-.304857 2.67198-.502117V-.980324H3.371357ZM2.175841-3.383313V-1.21345H.532005L2.175841-3.383313Z'/> +<path id='g5-40' d='M3.88543 2.905106C3.88543 2.86924 3.88543 2.84533 3.682192 2.642092C2.486675 1.43462 1.817186-.537983 1.817186-2.976837C1.817186-5.296139 2.379078-7.292653 3.765878-8.703362C3.88543-8.810959 3.88543-8.834869 3.88543-8.870735C3.88543-8.942466 3.825654-8.966376 3.777833-8.966376C3.622416-8.966376 2.642092-8.105604 2.056289-6.933998C1.446575-5.726526 1.171606-4.447323 1.171606-2.976837C1.171606-1.912827 1.338979-.490162 1.960648 .789041C2.666002 2.223661 3.646326 3.000747 3.777833 3.000747C3.825654 3.000747 3.88543 2.976837 3.88543 2.905106Z'/> +<path id='g5-41' d='M3.371357-2.976837C3.371357-3.88543 3.251806-5.36787 2.582316-6.75467C1.876961-8.18929 .896638-8.966376 .765131-8.966376C.71731-8.966376 .657534-8.942466 .657534-8.870735C.657534-8.834869 .657534-8.810959 .860772-8.607721C2.056289-7.400249 2.725778-5.427646 2.725778-2.988792C2.725778-.669489 2.163885 1.327024 .777086 2.737733C.657534 2.84533 .657534 2.86924 .657534 2.905106C.657534 2.976837 .71731 3.000747 .765131 3.000747C.920548 3.000747 1.900872 2.139975 2.486675 .968369C3.096389-.251059 3.371357-1.542217 3.371357-2.976837Z'/> +<path id='g5-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/> +<path id='g1-105' d='M2.375093-4.97335C2.375093-5.148692 2.247572-5.276214 2.064259-5.276214C1.857036-5.276214 1.625903-5.084932 1.625903-4.845828C1.625903-4.670486 1.753425-4.542964 1.936737-4.542964C2.14396-4.542964 2.375093-4.734247 2.375093-4.97335ZM1.211457-2.048319L.781071-.948443C.74122-.828892 .70137-.73325 .70137-.597758C.70137-.207223 1.004234 .079701 1.42665 .079701C2.199751 .079701 2.526526-1.036115 2.526526-1.139726C2.526526-1.219427 2.462765-1.243337 2.406974-1.243337C2.311333-1.243337 2.295392-1.187547 2.271482-1.107846C2.088169-.470237 1.761395-.143462 1.44259-.143462C1.346949-.143462 1.251308-.183313 1.251308-.398506C1.251308-.589788 1.307098-.73325 1.41071-.980324C1.490411-1.195517 1.570112-1.41071 1.657783-1.625903L1.904857-2.271482C1.976588-2.454795 2.072229-2.701868 2.072229-2.83736C2.072229-3.235866 1.753425-3.514819 1.346949-3.514819C.573848-3.514819 .239103-2.399004 .239103-2.295392C.239103-2.223661 .294894-2.191781 .358655-2.191781C.462267-2.191781 .470237-2.239601 .494147-2.319303C.71731-3.076463 1.083935-3.291656 1.323039-3.291656C1.43462-3.291656 1.514321-3.251806 1.514321-3.028643C1.514321-2.948941 1.506351-2.83736 1.42665-2.598257L1.211457-2.048319Z'/> +<path id='g1-106' d='M3.291656-4.97335C3.291656-5.124782 3.172105-5.276214 2.980822-5.276214C2.741719-5.276214 2.534496-5.053051 2.534496-4.845828C2.534496-4.694396 2.654047-4.542964 2.84533-4.542964C3.084433-4.542964 3.291656-4.766127 3.291656-4.97335ZM1.625903 .398506C1.506351 .884682 1.115816 1.40274 .629639 1.40274C.502117 1.40274 .382565 1.370859 .366625 1.362889C.613699 1.243337 .645579 1.028144 .645579 .956413C.645579 .765131 .502117 .661519 .334745 .661519C.103611 .661519-.111582 .860772-.111582 1.123786C-.111582 1.42665 .183313 1.625903 .637609 1.625903C1.123786 1.625903 2.000498 1.323039 2.239601 .366625L2.956912-2.486675C2.980822-2.582316 2.996762-2.646077 2.996762-2.765629C2.996762-3.203985 2.646077-3.514819 2.183811-3.514819C1.338979-3.514819 .844832-2.399004 .844832-2.295392C.844832-2.223661 .900623-2.191781 .964384-2.191781C1.052055-2.191781 1.060025-2.215691 1.115816-2.335243C1.354919-2.885181 1.761395-3.291656 2.1599-3.291656C2.327273-3.291656 2.422914-3.180075 2.422914-2.917061C2.422914-2.805479 2.399004-2.693898 2.375093-2.582316L1.625903 .398506Z'/> +<path id='g2-102' d='M5.332005-4.805978C5.571108-4.805978 5.66675-4.805978 5.66675-5.033126C5.66675-5.152677 5.571108-5.152677 5.355915-5.152677H4.387547C4.614695-6.38406 4.782067-7.232877 4.877709-7.615442C4.94944-7.902366 5.200498-8.177335 5.511333-8.177335C5.762391-8.177335 6.01345-8.069738 6.133001-7.962142C5.66675-7.914321 5.523288-7.567621 5.523288-7.364384C5.523288-7.12528 5.702615-6.981818 5.929763-6.981818C6.168867-6.981818 6.527522-7.185056 6.527522-7.639352C6.527522-8.141469 6.025405-8.416438 5.499377-8.416438C4.985305-8.416438 4.483188-8.033873 4.244085-7.567621C4.028892-7.149191 3.90934-6.718804 3.634371-5.152677H2.833375C2.606227-5.152677 2.486675-5.152677 2.486675-4.937484C2.486675-4.805978 2.558406-4.805978 2.797509-4.805978H3.56264C3.347447-3.694147 2.857285-.992279 2.582316 .286924C2.379078 1.327024 2.199751 2.199751 1.601993 2.199751C1.566127 2.199751 1.219427 2.199751 1.004234 1.972603C1.613948 1.924782 1.613948 1.398755 1.613948 1.3868C1.613948 1.147696 1.43462 1.004234 1.207472 1.004234C.968369 1.004234 .609714 1.207472 .609714 1.661768C.609714 2.175841 1.135741 2.438854 1.601993 2.438854C2.82142 2.438854 3.323537 .251059 3.455044-.3467C3.670237-1.267248 4.25604-4.447323 4.315816-4.805978H5.332005Z'/> +<path id='g2-112' d='M.514072 1.518306C.430386 1.876961 .382565 1.972603-.107597 1.972603C-.251059 1.972603-.37061 1.972603-.37061 2.199751C-.37061 2.223661-.358655 2.319303-.227148 2.319303C-.071731 2.319303 .095641 2.295392 .251059 2.295392H.765131C1.016189 2.295392 1.625903 2.319303 1.876961 2.319303C1.948692 2.319303 2.092154 2.319303 2.092154 2.10411C2.092154 1.972603 2.008468 1.972603 1.80523 1.972603C1.255293 1.972603 1.219427 1.888917 1.219427 1.793275C1.219427 1.649813 1.75741-.406476 1.829141-.681445C1.960648-.3467 2.283437 .119552 2.905106 .119552C4.25604 .119552 5.71457-1.637858 5.71457-3.395268C5.71457-4.495143 5.092902-5.272229 4.196264-5.272229C3.431133-5.272229 2.785554-4.531009 2.654047-4.363636C2.558406-4.961395 2.092154-5.272229 1.613948-5.272229C1.267248-5.272229 .992279-5.104857 .765131-4.65056C.549938-4.220174 .382565-3.490909 .382565-3.443088S.430386-3.335492 .514072-3.335492C.609714-3.335492 .621669-3.347447 .6934-3.622416C.872727-4.327771 1.099875-5.033126 1.578082-5.033126C1.853051-5.033126 1.948692-4.841843 1.948692-4.483188C1.948692-4.196264 1.912827-4.076712 1.865006-3.861519L.514072 1.518306ZM2.582316-3.730012C2.666002-4.064757 3.000747-4.411457 3.19203-4.578829C3.323537-4.698381 3.718057-5.033126 4.172354-5.033126C4.698381-5.033126 4.937484-4.507098 4.937484-3.88543C4.937484-3.311582 4.60274-1.960648 4.303861-1.338979C4.004981-.6934 3.455044-.119552 2.905106-.119552C2.092154-.119552 1.960648-1.147696 1.960648-1.195517C1.960648-1.231382 1.984558-1.327024 1.996513-1.3868L2.582316-3.730012Z'/> +<path id='g2-119' d='M4.112578-.729265C4.375592-.011955 5.116812 .119552 5.571108 .119552C6.479701 .119552 7.017684-.669489 7.352428-1.494396C7.627397-2.187796 8.069738-3.765878 8.069738-4.471233C8.069738-5.200498 7.699128-5.272229 7.591532-5.272229C7.304608-5.272229 7.041594-4.985305 7.041594-4.746202C7.041594-4.60274 7.12528-4.519054 7.185056-4.471233C7.292653-4.363636 7.603487-4.040847 7.603487-3.419178C7.603487-3.012702 7.268742-1.888917 7.017684-1.327024C6.682939-.597758 6.240598-.119552 5.618929-.119552C4.94944-.119552 4.734247-.621669 4.734247-1.171606C4.734247-1.518306 4.841843-1.936737 4.889664-2.139975L5.391781-4.148443C5.451557-4.387547 5.559153-4.805978 5.559153-4.853798C5.559153-5.033126 5.415691-5.152677 5.236364-5.152677C4.889664-5.152677 4.805978-4.853798 4.734247-4.566874C4.614695-4.100623 4.112578-2.080199 4.064757-1.817186C4.016936-1.613948 4.016936-1.482441 4.016936-1.219427C4.016936-.932503 3.634371-.478207 3.622416-.454296C3.490909-.32279 3.299626-.119552 2.929016-.119552C1.984558-.119552 1.984558-1.016189 1.984558-1.219427C1.984558-1.601993 2.068244-2.12802 2.606227-3.550685C2.749689-3.921295 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.920548-4.60274 1.3868-5.033126 1.829141-5.033126C1.948692-5.033126 2.139975-5.021171 2.139975-4.638605C2.139975-4.578829 2.139975-4.327771 1.936737-3.801743C1.374844-2.307347 1.255293-1.817186 1.255293-1.362889C1.255293-.107597 2.283437 .119552 2.893151 .119552C3.096389 .119552 3.634371 .119552 4.112578-.729265Z'/> +</defs> +<g id='page1' transform='matrix(1.13 0 0 1.13 -63.565683 -60.880618)'> +<use x='56.413267' y='73.409103' xlink:href='#g2-112'/> +<use x='62.28841' y='73.409103' xlink:href='#g5-40'/> +<use x='66.840736' y='73.409103' xlink:href='#g2-119'/> +<use x='75.208576' y='75.202366' xlink:href='#g1-105'/> +<use x='78.589848' y='73.409103' xlink:href='#g5-41'/> +<use x='86.463003' y='73.409103' xlink:href='#g5-61'/> +<use x='118.724909' y='65.321344' xlink:href='#g2-102'/> +<use x='125.771346' y='65.321344' xlink:href='#g5-40'/> +<use x='130.323672' y='65.321344' xlink:href='#g2-119'/> +<use x='138.691512' y='67.114607' xlink:href='#g1-105'/> +<use x='142.072784' y='65.321344' xlink:href='#g5-41'/> +<use x='147.820623' y='57.650464' xlink:href='#g3-51'/> +<rect x='147.820623' y='58.737042' height='.358656' width='3.652912'/> +<use x='147.820623' y='64.034498' xlink:href='#g3-52'/> +<rect x='100.083997' y='70.181217' height='.478187' width='71.724077'/> +<use x='100.083997' y='74.650602' xlink:href='#g0-80'/> +<use x='112.703377' y='77.805443' xlink:href='#g4-49'/> +<use x='116.93756' y='77.805443' xlink:href='#g4-48'/> +<use x='121.171743' y='77.805443' xlink:href='#g4-48'/> +<use x='125.405926' y='77.805443' xlink:href='#g4-48'/> +<use x='129.640108' y='77.805443' xlink:href='#g4-48'/> +<use x='112.703377' y='87.104054' xlink:href='#g1-106'/> +<use x='116.587402' y='87.104054' xlink:href='#g4-61'/> +<use x='123.173909' y='87.104054' xlink:href='#g4-49'/> +<use x='136.364921' y='83.617054' xlink:href='#g2-102'/> +<use x='143.411357' y='83.617054' xlink:href='#g5-40'/> +<use x='147.963683' y='83.617054' xlink:href='#g2-119'/> +<use x='156.331523' y='85.410317' xlink:href='#g1-106'/> +<use x='160.71368' y='83.617054' xlink:href='#g5-41'/> +<use x='166.46152' y='75.946174' xlink:href='#g3-51'/> +<rect x='166.46152' y='77.032752' height='.358656' width='3.652912'/> +<use x='166.46152' y='82.330208' xlink:href='#g3-52'/> +</g> +</svg> \ No newline at end of file diff --git a/_resources/neural-lm-embedding.png b/_resources/neural-lm-embedding.png new file mode 100644 index 0000000..a2056b8 Binary files /dev/null and b/_resources/neural-lm-embedding.png differ diff --git a/_resources/neural-network-2-layer-forward.png b/_resources/neural-network-2-layer-forward.png new file mode 100644 index 0000000..523bdba Binary files /dev/null and b/_resources/neural-network-2-layer-forward.png differ diff --git a/_resources/neural-network-2-layer.png b/_resources/neural-network-2-layer.png new file mode 100644 index 0000000..efd776d Binary files /dev/null and b/_resources/neural-network-2-layer.png differ diff --git a/_resources/neural-network.svg b/_resources/neural-network.svg new file mode 100644 index 0000000..fd0f9f0 --- /dev/null +++ b/_resources/neural-network.svg @@ -0,0 +1,291 @@ +<svg id="Layer_1" data-name="Layer 1" xmlns="http://www.w3.org/2000/svg" width="589" height="283" viewBox="0 0 589 283"> + <defs> + <style> + .cls-1 { + fill: #484748; + } + + .cls-2 { + fill: #fff; + } + </style> + </defs> + <title>deeplearning + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/_resources/neural-style-transfer.png b/_resources/neural-style-transfer.png new file mode 100644 index 0000000..0404513 Binary files /dev/null and b/_resources/neural-style-transfer.png differ diff --git a/_resources/nn-computation.png b/_resources/nn-computation.png new file mode 100644 index 0000000..032d662 Binary files /dev/null and b/_resources/nn-computation.png differ diff --git a/_resources/nn-example.png b/_resources/nn-example.png new file mode 100644 index 0000000..a98dde3 Binary files /dev/null and b/_resources/nn-example.png differ diff --git a/_resources/nn-representation.png b/_resources/nn-representation.png new file mode 100644 index 0000000..3312edb Binary files /dev/null and b/_resources/nn-representation.png differ diff --git a/_resources/nn_frame.png b/_resources/nn_frame.png new file mode 100644 index 0000000..717ce71 Binary files /dev/null and b/_resources/nn_frame.png differ diff --git a/_resources/object-anchorbox.png b/_resources/object-anchorbox.png new file mode 100644 index 0000000..f0681e0 Binary files /dev/null and b/_resources/object-anchorbox.png differ diff --git a/_resources/object-clf-detect.png b/_resources/object-clf-detect.png new file mode 100644 index 0000000..f663efb Binary files /dev/null and b/_resources/object-clf-detect.png differ diff --git a/_resources/object-clf-local-y.png b/_resources/object-clf-local-y.png new file mode 100644 index 0000000..71d83f0 Binary files /dev/null and b/_resources/object-clf-local-y.png differ diff --git a/_resources/object-clf-local.png b/_resources/object-clf-local.png new file mode 100644 index 0000000..cc61a8d Binary files /dev/null and b/_resources/object-clf-local.png differ diff --git a/_resources/object-iou.png b/_resources/object-iou.png new file mode 100644 index 0000000..b27f1f2 Binary files /dev/null and b/_resources/object-iou.png differ diff --git a/_resources/object-landmark.png b/_resources/object-landmark.png new file mode 100644 index 0000000..24364aa Binary files /dev/null and b/_resources/object-landmark.png differ diff --git a/_resources/object-nonmax.png b/_resources/object-nonmax.png new file mode 100644 index 0000000..5ade5d0 Binary files /dev/null and b/_resources/object-nonmax.png differ diff --git a/_resources/object-slide-window.png b/_resources/object-slide-window.png new file mode 100644 index 0000000..4ce1556 Binary files /dev/null and b/_resources/object-slide-window.png differ diff --git a/_resources/object-sliding-conv.png b/_resources/object-sliding-conv.png new file mode 100644 index 0000000..8232d2f Binary files /dev/null and b/_resources/object-sliding-conv.png differ diff --git a/_resources/object-sliding-conv2.png b/_resources/object-sliding-conv2.png new file mode 100644 index 0000000..177e0ac Binary files /dev/null and b/_resources/object-sliding-conv2.png differ diff --git a/_resources/object-yolo-alg.png b/_resources/object-yolo-alg.png new file mode 100644 index 0000000..7e45272 Binary files /dev/null and b/_resources/object-yolo-alg.png differ diff --git a/_resources/object-yolo-algorithm.png b/_resources/object-yolo-algorithm.png new file mode 100644 index 0000000..bbe571a Binary files /dev/null and b/_resources/object-yolo-algorithm.png differ diff --git a/_resources/onpremise.png b/_resources/onpremise.png new file mode 100644 index 0000000..26d5d6a Binary files /dev/null and b/_resources/onpremise.png differ diff --git a/_resources/prob-conditional.svg b/_resources/prob-conditional.svg new file mode 100644 index 0000000..2f9b541 --- /dev/null +++ b/_resources/prob-conditional.svg @@ -0,0 +1,44 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/prob-cost.svg b/_resources/prob-cost.svg new file mode 100644 index 0000000..133df8a --- /dev/null +++ b/_resources/prob-cost.svg @@ -0,0 +1,64 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/reg-logistic-cost.svg b/_resources/reg-logistic-cost.svg new file mode 100644 index 0000000..e9dc980 --- /dev/null +++ b/_resources/reg-logistic-cost.svg @@ -0,0 +1,71 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/reg-logistic-l1.svg b/_resources/reg-logistic-l1.svg new file mode 100644 index 0000000..3bbe03c --- /dev/null +++ b/_resources/reg-logistic-l1.svg @@ -0,0 +1,33 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/reg-logistic-l2.svg b/_resources/reg-logistic-l2.svg new file mode 100644 index 0000000..60a96ad --- /dev/null +++ b/_resources/reg-logistic-l2.svg @@ -0,0 +1,38 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/reg-nn-cost.svg b/_resources/reg-nn-cost.svg new file mode 100644 index 0000000..b9620bb --- /dev/null +++ b/_resources/reg-nn-cost.svg @@ -0,0 +1,105 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/reg-nn-fnorm.svg b/_resources/reg-nn-fnorm.svg new file mode 100644 index 0000000..697f7dd --- /dev/null +++ b/_resources/reg-nn-fnorm.svg @@ -0,0 +1,67 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/reg-nn-grad.svg b/_resources/reg-nn-grad.svg new file mode 100644 index 0000000..905dc60 --- /dev/null +++ b/_resources/reg-nn-grad.svg @@ -0,0 +1,50 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/reg-nn-wdecay.svg b/_resources/reg-nn-wdecay.svg new file mode 100644 index 0000000..4b8d9fb --- /dev/null +++ b/_resources/reg-nn-wdecay.svg @@ -0,0 +1,99 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/relu.png b/_resources/relu.png new file mode 100644 index 0000000..83937bb Binary files /dev/null and b/_resources/relu.png differ diff --git a/_resources/resnet-block.png b/_resources/resnet-block.png new file mode 100644 index 0000000..eded245 Binary files /dev/null and b/_resources/resnet-block.png differ diff --git a/_resources/resnet-network.png b/_resources/resnet-network.png new file mode 100644 index 0000000..74267f3 Binary files /dev/null and b/_resources/resnet-network.png differ diff --git a/_resources/resnet-plain-34.png b/_resources/resnet-plain-34.png new file mode 100644 index 0000000..dc5a627 Binary files /dev/null and b/_resources/resnet-plain-34.png differ diff --git a/_resources/resnet-resnet-34.png b/_resources/resnet-resnet-34.png new file mode 100644 index 0000000..2d83778 Binary files /dev/null and b/_resources/resnet-resnet-34.png differ diff --git a/_resources/resnet.png b/_resources/resnet.png new file mode 100644 index 0000000..eb640bc Binary files /dev/null and b/_resources/resnet.png differ diff --git a/_resources/rnn-forward.png b/_resources/rnn-forward.png new file mode 100644 index 0000000..fffd996 Binary files /dev/null and b/_resources/rnn-forward.png differ diff --git a/_resources/rnn-lm.png b/_resources/rnn-lm.png new file mode 100644 index 0000000..4174045 Binary files /dev/null and b/_resources/rnn-lm.png differ diff --git a/_resources/rnn-notation.png b/_resources/rnn-notation.png new file mode 100644 index 0000000..adbd944 Binary files /dev/null and b/_resources/rnn-notation.png differ diff --git a/_resources/rnn-type.png b/_resources/rnn-type.png new file mode 100644 index 0000000..bae882b Binary files /dev/null and b/_resources/rnn-type.png differ diff --git a/_resources/rnn-unit.png b/_resources/rnn-unit.png new file mode 100644 index 0000000..35d9262 Binary files /dev/null and b/_resources/rnn-unit.png differ diff --git a/_resources/saddle-point.png b/_resources/saddle-point.png new file mode 100644 index 0000000..4af3c2d Binary files /dev/null and b/_resources/saddle-point.png differ diff --git a/_resources/sentiment-model-rnn.png b/_resources/sentiment-model-rnn.png new file mode 100644 index 0000000..d08cac1 Binary files /dev/null and b/_resources/sentiment-model-rnn.png differ diff --git a/_resources/sentiment-model-simple.png b/_resources/sentiment-model-simple.png new file mode 100644 index 0000000..1d8d44c Binary files /dev/null and b/_resources/sentiment-model-simple.png differ diff --git a/_resources/seq-seq.png b/_resources/seq-seq.png new file mode 100644 index 0000000..179e908 Binary files /dev/null and b/_resources/seq-seq.png differ diff --git a/_resources/siamese-network.png b/_resources/siamese-network.png new file mode 100644 index 0000000..5cb354c Binary files /dev/null and b/_resources/siamese-network.png differ diff --git a/_resources/sigmoid-latex.svg b/_resources/sigmoid-latex.svg new file mode 100644 index 0000000..6d335fd --- /dev/null +++ b/_resources/sigmoid-latex.svg @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/sigmoid.png b/_resources/sigmoid.png new file mode 100644 index 0000000..fd97675 Binary files /dev/null and b/_resources/sigmoid.png differ diff --git a/_resources/softmax.svg b/_resources/softmax.svg new file mode 100644 index 0000000..b41ccd2 --- /dev/null +++ b/_resources/softmax.svg @@ -0,0 +1,56 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/spark_toolset-1.png b/_resources/spark_toolset-1.png new file mode 100644 index 0000000..da7f75a Binary files /dev/null and b/_resources/spark_toolset-1.png differ diff --git a/_resources/spark_toolset.png b/_resources/spark_toolset.png new file mode 100644 index 0000000..da7f75a Binary files /dev/null and b/_resources/spark_toolset.png differ diff --git a/_resources/speech-recognition-attention.png b/_resources/speech-recognition-attention.png new file mode 100644 index 0000000..89917e3 Binary files /dev/null and b/_resources/speech-recognition-attention.png differ diff --git a/_resources/style-cost1.png b/_resources/style-cost1.png new file mode 100644 index 0000000..37da752 Binary files /dev/null and b/_resources/style-cost1.png differ diff --git a/_resources/style-cost2.png b/_resources/style-cost2.png new file mode 100644 index 0000000..2b051a7 Binary files /dev/null and b/_resources/style-cost2.png differ diff --git a/_resources/style-cost3.png b/_resources/style-cost3.png new file mode 100644 index 0000000..b6f5c78 Binary files /dev/null and b/_resources/style-cost3.png differ diff --git a/_resources/tanh-latex.svg b/_resources/tanh-latex.svg new file mode 100644 index 0000000..1e32bb5 --- /dev/null +++ b/_resources/tanh-latex.svg @@ -0,0 +1,30 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_resources/tanh.png b/_resources/tanh.png new file mode 100644 index 0000000..358b530 Binary files /dev/null and b/_resources/tanh.png differ diff --git a/_resources/theory.broadcast_2.gif b/_resources/theory.broadcast_2.gif new file mode 100644 index 0000000..163a847 Binary files /dev/null and b/_resources/theory.broadcast_2.gif differ diff --git a/_resources/transfer-learning.png b/_resources/transfer-learning.png new file mode 100644 index 0000000..bebaf84 Binary files /dev/null and b/_resources/transfer-learning.png differ diff --git a/_resources/tree-def.png b/_resources/tree-def.png new file mode 100644 index 0000000..904ac55 Binary files /dev/null and b/_resources/tree-def.png differ diff --git a/_resources/tree-eval-derivs.png b/_resources/tree-eval-derivs.png new file mode 100644 index 0000000..716702a Binary files /dev/null and b/_resources/tree-eval-derivs.png differ diff --git a/_resources/trigger-word-detection.png b/_resources/trigger-word-detection.png new file mode 100644 index 0000000..6a854b9 Binary files /dev/null and b/_resources/trigger-word-detection.png differ diff --git a/_resources/vgg-16.png b/_resources/vgg-16.png new file mode 100644 index 0000000..42b64bb Binary files /dev/null and b/_resources/vgg-16.png differ diff --git a/_resources/visualizing-nn.png b/_resources/visualizing-nn.png new file mode 100644 index 0000000..27f7774 Binary files /dev/null and b/_resources/visualizing-nn.png differ diff --git a/_resources/word-embedding.png b/_resources/word-embedding.png new file mode 100644 index 0000000..f0d7474 Binary files /dev/null and b/_resources/word-embedding.png differ