Machine learning vs human learning

Language (English, Croatian, Estonian - all 'Natural Languages') is the most powerfull tool created by Humanity. Languge/communication rised humans over all other animals and made us 'the Kings of our Planet'

But into our human communications has entered a new powerful participant - computer

On Internet, we often speak with computers - believing that our conversation partner is human

Computers can already create new texts/paintings/songs which are indistinguishable from human-created ones

Many text analyzing and classication tasks (spam and plaguarism, hate speech e.g. detection) are routinely handled by programs

Natural Language Processing (NLP) and Natural Language Generation (NLG) are increasingly used

Teachers and students should understand the basics of those technologies

NLP begins with creation of massive text corpus - all produced strings are statistically inferred from this corpus

Here were used texts from MIPRO CE presentations 2017..2020 (made open-source by Mipro)

NLP algorithms work well only with 'clean' text - no tables, no pictures, no numbers, no links (often is removed also punctuation and text converted to lowercase); all information about authors, affiliations, keywords, acknowledgements etc should be removed.

Text cleaning is a formidable task. Here the .pdf was converted to .doc and then used Microsoft Word macro language.

Text cleaning involves lot of manual work:
"carbage in -> carbage out"

Text may be handled character level, on word level, on paragraph/document level ...

Word level requires several additional operations (stopword removal, lemmetization stemming), therefore here new text generation was done on character level

Next were counted frequencies of words; in both corpuses the most frequent word was the same:

The most frequent words in both corpuses overlap - authors were dicussing the same topics

The common tasks in NLP are

For this are considered entropy of text and subsets of words occuring close to each other - the consequtive words (n-grams) and non-consequtive (BOW - Bags Of Words)

n-gram of a word is a sequency of consequitive n words containg the word

E.g. 3-grams for the word students in text Students like games. Students seek educational games (after converting text to lowercase and removing punctuation) are:
students like games, like games students, games students seek, students seek educational

Entropy shows average amount of information of words (in bits).
The Croatian language is more complex than English: more letters, more word forms etc., thus its entropy is greater, but decreases with bigger phrases. Entropy of single words (unigrams), two consecutive words (bigrams), three consecutive words (trigrams) etc. shows essential difference between English and Croatian languages.

Phrases can be separated on their occurrence spectrum:
- significant phrases occur in tight 'dumplings' which are unevenly distributed
- common/insignificant phrases are evenly distributed in the whole text

Spectrums of words 'Table', 'Tablica', the percentage sign % show, that the MiproCE is actually already a Data Science conference:

The most fashinating feature of NLP is creation of new texts

If we already have a sequence of words
$${w_1},{w_2},...,{w_{i - 2}},{w_{i - 1}}$$ then the probability of the next word $$P({w_i}|{w_1},...,{w_{i - 1}})$$ could be estimated by frequences of similar situations: $$P({w_i}|{w_1},...,{w_{i - 1}}) \approx \frac{{Fr({w_1},...,{w_{i - 1}},{w_i})}}{{Fr({w_1},...,{w_{i - 1}})}}$$

This is the key idea of NLP and Machine Learning (ML) in NLP

A word vector encodes frequences for other words to occure in its neighbourhood
The text corpus is considered as a vector spase, whose dimension is equal to the number of words in vocabulary

Thus it becomes possible to consider distance between words and present words as points in thie high-dimensional space

Program is able to create new texts resembling human-created one:

Students have the main factor in closed computer literacy in the environment and society. Students can be applied on the tool, such as the most out of expertise, they are also addressed through the profile of the equipment and society of digital technology that is more important than the students.

But program created also total nonsense:
Students are not included in the process of e- learning in the context of the process of teaching and learning.

NLP has nothing in common with traditional linguistic concepts (grammar, parsing, parts of speech) and linguistic theory, it may be called "linquistics without language theory"

NLP is well characterized by the statement from Frederick Jelinek, long-time head of the Center for Language and Speech Processing at Johns Hopkins University:
"Whenever I fire a linguist our system performance improves"

The 'learning' in ML does not change behavior of the computer (if it does, then the computer is severely broken)

The 'learning' in ML is similar to calculating averages

But NLP and ML can be used and are increasingly used for simple natural-language tasks, e.g. plagiarism discovery

Thus teachers and students should understand the basics of these techniques

That's ALL !

Available on-line at 'http://www.ttu.ee/staff/~jaak.henno/Mipro2021/'

Let's all cross fingers for a NORMAL MIPRO sometimes again!