Spark package

Spark – LDA : A Complete example of clustering algorithm for topic discovery.

October 8, 2016October 12, 2016Apache Spark, Spark, Studio-ScalaApache Spark, cleaning data, clustering, clustering in distributed systems, distributed LDA, document Clustering, full example, introduction to machine learning, LDA algorithm, lemmatization, Machine Learning, Machine Learning Pipeline, Natural language processing, NLP, opensource, Spark, Spark package, stanford nlp, Stemming, Stopwords removal, Testing, tokenization, Topic identification

Reading Time: 6 minutes In this blog we will be demonstrating the functionality of applying the full ML pipeline over a set of documents which in this case we are using 10 books from the internet. So lets start with first thing first.. What is Clustering ? Clustering is the task of grouping a set of objects in such a way that objects in the same group (called a Continue Reading