tag:blogger.com,1999:blog-37026881973928967992024-03-05T16:35:45.124-08:00Data ScienceRead Real World Expertise and Analysis on Applied Data Science, AI, Big Data and Analytics Authored by Jai Krishna Ponnappan.Jai Krishna Ponnappanhttp://www.blogger.com/profile/02699917895019277960noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-3702688197392896799.post-2224797351629156162021-09-16T09:30:00.003-07:002021-09-16T09:34:25.351-07:00Data Science Applications - Page Rank Text Summarization <p><span style="font-family: Baskervville; font-size: medium;"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4dCRIVudeRsoWuejJdiW7gBJQ8z-kHnI13MqnJijQetCHcQhvbvkWvHOK40-REQ4ReFrKMTWZfDjB_Emr2JjChzAmOl2AMHH-3GoOs2SrfiXGqqesIAU99ONpOVZPLYb7M5jiouo7oGs/s612/Data-Science-PageRank-Algorithms-Jai-Krishna-Ponnappan-Web-1.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="344" data-original-width="612" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4dCRIVudeRsoWuejJdiW7gBJQ8z-kHnI13MqnJijQetCHcQhvbvkWvHOK40-REQ4ReFrKMTWZfDjB_Emr2JjChzAmOl2AMHH-3GoOs2SrfiXGqqesIAU99ONpOVZPLYb7M5jiouo7oGs/w400-h225/Data-Science-PageRank-Algorithms-Jai-Krishna-Ponnappan-Web-1.jpg" width="400" /></a></div><br /><span style="font-family: Baskervville; font-size: medium;"><br />
</span><h2 style="text-align: left;"><span style="color: #ffd966; font-family: Baskervville; font-size: medium;">What Is Text Summarization?</span></h2><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-spacerun: yes;"><span style="font-family: Baskervville; font-size: medium;"> </span></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">Text summarization is
the most significant application in natural language processing. </span></h3><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">It assists with reducing the quantity of original text and
extracting just the relevant information. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">The technique of text summarizing is also known as data
reduction. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">It entails generating an outline of the original text that
allows the user to get key bits of information from that text in a much shorter
amount of time. <o:p></o:p></span></p>
<span style="font-family: Baskervville; font-size: medium;"><br />
</span><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><h2 style="text-align: left;"><span style="color: #ffd966; font-family: Baskervville; font-size: medium;">Text Summarization Processes Types</span></h2><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">The process of text
summarizing may be categorized in many ways, including: The classification of
the text summarizing process is shown in Figure. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">As demonstrated, text summarization may be categorized into
many categories, each of which can be further subdivided. <o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p>
<h3 style="text-align: left;"><span style="color: #ffd966; font-family: Baskervville; font-size: medium;">Depending on the number of documents Text summary is further
divided into categories depending on the number of pages in a document: </span></h3><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><h4 style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">•
Single: </span></h4><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;"><div style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">Because the outline is short, clear, and concise, it becomes more
important. </span></div><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">Some subdocuments may be combined to form a single document.
</span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">They may be created out of certain subdocuments' documents
that place unusual emphasis on different viewpoints, despite the fact that
these reports all cover the same topic. </span></p></blockquote><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p>
<h4 style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">• Several: </span></h4><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;"><div style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">A multi-document summary is a technique for
managing a large amount of data in multiple linked supply documents by
including just the most important information or main concepts in a little
amount of space. </span></div><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">A multi-document report has recently become a hot topic in
automated summarization.</span></p></blockquote><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p>
<h4 style="text-align: left;"><b><span style="color: #ffd966; font-family: Baskervville; font-size: medium;">A. Based on the Usage Summary Text summarization may be further
subdivided into the following categories depending on summary usage: </span></b></h4><div><b><span style="font-family: Baskervville; font-size: medium;"><br /></span></b></div><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;"><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">• Generic
Summaries: Generic summaries do not target any specific cluster since they are
written for a large audience.</span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">• Query-based: Query-based or subject-focused inquiries are
tailored to an individual's or a group's unique requirements and address a
single issue.</span></p></blockquote><p><span style="font-family: Baskervville; font-size: medium;"> </span></p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;"><p class="MsoNormal"> </p><p class="MsoNormal"><b><span style="font-family: Baskervville; font-size: medium;">The goal of query-based text summarization is to extract
fundamental information from the original text that answers the question. </span></b></p><p class="MsoNormal"><b><span style="font-family: Baskervville; font-size: medium;">The proper response is presented in a small, predetermined
amount of words. </span></b></p></blockquote><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p>
<h4 style="text-align: left;"><span style="color: #ffd966; font-family: Baskervville; font-size: medium;">B. Techniques-based Text summarization may be further divided
into subcategories based on the following techniques: </span></h4><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><h4 style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">• Supervised: </span></h4><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><p class="MsoNormal"></p><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">Supervised
text summarization is similar to supervised key extraction in that it is
supervised.</span></li></ul><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">Essentially, if you have a collection of documents and
human-generated summaries for them, you can learn the characteristics of
phrases that make them a good fit for inclusion in the summary. </span></li></ul><p></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p>
<h4 style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">• Unsupervised: </span></h4><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><p class="MsoNormal"></p><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">The use of unsupervised key extraction
eliminates the need for training data.</span></li></ul><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">It approaches the problem from a different perspective.</span></li></ul><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">Rather of trying to learn explicit characteristics that
characterize important words, the TextRank algorithm takes use of the content's
structure to choose key phrases that seem "central" to the text,
similar to how PageRank selects major websites. </span></li></ul><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span><p></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><h4 style="text-align: left;"><span style="color: #ffd966; font-family: Baskervville; font-size: medium;">C. Based on the Textual Characteristics of the Summary Text
summarization may be classified into a variety of groups depending on the
features of the summary text, such as:</span></h4><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><h4 style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">• Abstractive Summarization: </span></h4><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><p class="MsoNormal"></p><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">Abstractive
summarization methods change the material by adding new phrases, rephrasing, or
inserting terms not found in the original text.</span></li></ul><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">For a flawless abstractive summary, the model must first
understand the text before expressing it with new words and phrases.</span></li></ul><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">Complex elements like as generalization, paraphrase, and
integrating real-world information are included. </span></li></ul><p></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p>
<h4 style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">• Extractive Summarization:</span></h4><span style="font-family: Baskervville; font-size: medium;"><br />
</span><p class="MsoNormal"></p><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">Summarization creates summaries by combining various
portions of phrases taken from the source material.</span></li></ul><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">In such situations, rating the importance of different
phrases is often a major improvement.</span></li></ul><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">A selection of essential data is extracted and then
reassembled to provide a summary.</span></li></ul><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span><p></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p><span style="font-family: Baskervville; font-size: medium;"> </span></o:p></p><p class="MsoNormal"><o:p><span style="font-family: Baskervville; font-size: medium;"><br /></span></o:p></p><p class="MsoNormal"><o:p><span style="font-family: Baskervville; font-size: medium;"><br /></span></o:p></p><p class="MsoNormal"><o:p><span style="font-family: Baskervville; font-size: medium;"><br /></span></o:p></p>
<h2 style="text-align: left;"><span style="color: #ffd966; font-family: Baskervville; font-size: medium;">Algorithm of PageRank.</span></h2><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">Around 1998, Page and
Brin collaborated to create and improve the PageRank set of criteria. It was primarily used in the prototype of Google's search
engine. </span></h3><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;">The purpose of this collection of criteria is to determine
the popularity, or importance, of a website based on the concept of web
interconnectivity. <o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p>
<h3 style="text-align: left;"><span style="font-family: Baskervville; font-size: medium;">According to the theory, a web page with more incoming
hyperlinks performs a larger function than a web page with fewer incoming
hyperlinks. </span></h3><div><span style="font-family: Baskervville; font-size: medium;"><br /></span></div><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"></p><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">A online page having a hyperlink from a web page considered
to be of extreme importance is also significant.</span></li></ul><ul style="text-align: left;"><li><span style="font-family: Baskervville; font-size: medium;">PageRank is one of the most widely used ranking algorithms,
and it was created as a method for analyzing Weblinks.</span></li></ul><ul style="text-align: left;"><li><b><span style="font-family: Baskervville; font-size: medium;">The PageRank algorithm is used to calculate the weight of
online pages, and it is the same concept that Google uses to give a rank to a
web page based on a search result. </span></b></li></ul><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span><p></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville; font-size: medium;"><br /></span></p>
<p class="MsoNormal"><o:p><span style="font-family: Baskervville; font-size: medium;"> </span></o:p></p>Jai Krishna Ponnappanhttp://www.blogger.com/profile/02699917895019277960noreply@blogger.comtag:blogger.com,1999:blog-3702688197392896799.post-40285979325874214452021-06-01T05:11:00.000-07:002021-06-01T05:11:15.584-07:00Data Science and Statistics <div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNrl7GR9K5GtyE_Felunyzs_OiOnB8sFuNrknI34P4mkJ8_HZfbGyBIIkXn6E7YsFBei5jbBUSWNET9EYOJ79-bG7-fwn4_LdMFVcBWklt83IT1FEmf-343_R9-ux3kTOhGTK3GQqP_cQ/s340/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-10.webp" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: Baskervville;"><img border="0" data-original-height="340" data-original-width="340" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNrl7GR9K5GtyE_Felunyzs_OiOnB8sFuNrknI34P4mkJ8_HZfbGyBIIkXn6E7YsFBei5jbBUSWNET9EYOJ79-bG7-fwn4_LdMFVcBWklt83IT1FEmf-343_R9-ux3kTOhGTK3GQqP_cQ/w400-h400/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-10.webp" width="400" /></span></a></div><span style="font-family: Baskervville;"><br /></span><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;">The statistical ideas utilized in data science are explained in this article. It's crucial to note that data science isn't a new idea, and that most statisticians are capable of working as data scientists. </span></p><p><span style="font-family: Baskervville;">Many principles from statistics are used in data science since statistics is the finest instrument for processing and interpreting data. Statistical techniques may help you extract a lot of information from the data collection. </span></p><p><b><span style="font-family: Baskervville;">You should study everything you can about statistics if you want to understand data science and become an expert in the area. </span></b></p><p><b><span style="font-family: Baskervville;">While there are numerous areas in statistics that a data scientist should be familiar with, the following are the most crucial: </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p></p><ol style="text-align: left;"><li><span style="font-family: Baskervville;"><b>Descriptive Statistics </b>are a type of statistical analysis that is used to describe something. </span></li><li><span style="font-family: Baskervville;"><b>Inferential Statistics</b> are statistics that may be used to make inferences. </span></li></ol><div><span style="font-family: Baskervville;"><br /></span></div><p></p><p><span style="font-family: Baskervville;">Descriptive statistics is the act of describing or looking at data in a way that makes it easier to understand. </span></p><p><span style="font-family: Baskervville;">This technique aids in the quantitative summarizing of data using numerical representations or graphs. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">The following are some of the subjects you should study: </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p></p><ol style="text-align: left;"><li><b><span style="font-family: Baskervville;">Normal Distribution </span></b></li><li><b><span style="font-family: Baskervville;">Central Tendency </span></b></li><li><b><span style="font-family: Baskervville;">Variability </span></b></li><li><b><span style="font-family: Baskervville;">Kurtosis </span></b></li></ol><p></p><h3 style="text-align: left;"><span style="font-family: Baskervville;"><br /></span></h3><h3 style="text-align: left;"><span style="font-family: Baskervville;"><br />Normal Distribution </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">A normal distribution, also known as a Gaussian distribution, is a continuous distribution often used in statistics. Any data set following a normal distribution is spread across a graph, also a bell-shaped curve. </span></p><p><span style="font-family: Baskervville;">In normal distributions, the data points in the set peak at the center of the bell-shaped curve, which represents the center of the data set. </span></p><p><span style="font-family: Baskervville;">When the data moves away from the mean, it will fall to the end of the curve. You need to ensure the data you look at is distributed normally if you want to make inferences from the data set.</span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Central Tendency </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">Measures of central tendency aid in determining the data set's center values. The mean, median, and mode are the three most widely used measurements. Any distribution's mean, or arithmetic mean, is located at the middle of the data set. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">The following formula may be used to compute the data set's mean or average: </span></b></p><p><i><span style="font-family: Baskervville;">(the total number of points in the data collection) / (number of data points) </span></i></p><p><span style="font-family: Baskervville;">Another metric is the median, which is the midpoint of the data set when the points are sorted ascending. </span></p><p><span style="font-family: Baskervville;">You can easily calculate the midpoint if you have an odd amount of values, but if you have an even number of data points, you take the average of the two data points in the middle of the data set. </span></p><p><span style="font-family: Baskervville;">The mode is the last metric, and its value is the data point that appears the most times in the data collection. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Variability </span></h3><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;">Variability is a factor that aids in determining the distance between the data points in a data collection and the average or mean of the data points. </span></p><p><span style="font-family: Baskervville;">This number also displays the difference between the chosen data points. Variability may be viewed and assessed using central measure metrics such as range, variation, and standard deviation. </span></p><p><span style="font-family: Baskervville;">The range is a number that represents the difference between the data set's lowest and greatest values. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Skewness and Kurtosis </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">The skewness of a data collection might assist you figure out how symmetrical it is. The data set will take the shape of a bell curve if it is spread evenly. The data is not skewed if the curve is formed equally. </span></p><p><span style="font-family: Baskervville;">The data is negatively or positively skewed if the curve goes to the right or left side of the data points, respectively. This indicates that the data is dominating on either the left or right side of the central tendency measurements. </span></p><p><span style="font-family: Baskervville;">Kurtosis is a metric that aids in determining the distribution's tails. You can tell if the data is light or heavy-tailed by plotting the dots on a graph. Based on the center region of the distribution, you may make this assumption. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Statistical Inference </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p></p><ul style="text-align: left;"><li><span style="font-family: Baskervville;">Inferential statistics are used to get insights into a data collection. Descriptive statistics give information about the data. </span></li><li><span style="font-family: Baskervville;">Inferential statistics is concerned with drawing conclusions about a big population from a small sample of data. </span></li></ul><p></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;">Assume you're trying to figure out how many individuals in Africa have got the polio vaccination. </span></p><p><span style="font-family: Baskervville;">This analysis can be carried out in two ways: Inquire of every person in Africa if they have received the polio vaccination. Take a sample of people from throughout the continent, make sure they're from various sections, then extrapolate the results throughout the entire continent. </span></p><p><span style="font-family: Baskervville;">The first procedure is difficult, if not impossible, to accomplish. It's impossible to walk across the country asking people if they've got the vaccination. </span></p><p><span style="font-family: Baskervville;">The second technique is preferable since it allows you to draw conclusions or insights from the sample you've chosen and extrapolate the results to a larger population. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><h2 style="text-align: left;"><span style="font-family: Baskervville;">The following are some inferential statistics tools:</span></h2><div><span style="font-family: Baskervville;"><br /></span></div><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Theorem of the Central Limit </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">"The average of the sample equals the average of the total population," says the central limit theorem. This demonstrates that the sample and population have the same features and measurements of the data's central tendency, such as standard deviation. </span></p><p><span style="font-family: Baskervville;">This implies you may choose more data points, which will result in a normal curve. </span></p><p><span style="font-family: Baskervville;">If you wish to apply the central limit theorem, you must first grasp the idea of confidence intervals. This represents the population's mean value as a rough estimate. </span></p><p><span style="font-family: Baskervville;">The total of an error margin is used in the process of constructing an interval in the population. "Multiplying the standard error of the mean with the zscore of the percentage of confidence level" is one way to determine this mistake. </span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;"><br /></span></h3><h3 style="text-align: left;"><span style="font-family: Baskervville;">Testing Hypotheses </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">The extent to which you can test whatever assumption you make about the data set is known as hypothesis testing. You can collect the findings of your hypothesis analysis on a smaller population using this kind of testing. </span></p><p><span style="font-family: Baskervville;">The null hypothesis is the theory you'll be testing, and we'll be comparing it against the alternative hypothesis to see if it's valid. The case you must test is the null hypothesis. </span></p><p><span style="font-family: Baskervville;">Consider the following scenario: you're conducting a survey to find out who smokes and who doesn't, as well as how smokers are affected by cancer. </span></p><p><span style="font-family: Baskervville;">When conducting this survey, you make the assumption that the number of cancer patients who smoke is equal to the number of cancer patients who do not smoke. This is your null hypothesis, and you must test it in order to reject it. </span></p><p><span style="font-family: Baskervville;">The alternative hypothesis is that the number of cancer patients who smoke is higher than the number of cancer patients who do not. You may test hypotheses and evaluate data to see if the null hypothesis is valid or not using the data and evidence provided. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">ANOVA (Analysis of Variance) </span></h3><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;">Another statistical technique used to test hypotheses across several sets of data is ANOVA. This idea aids in determining if the groups under consideration have similar averages and variations. </span></p><p><span style="font-family: Baskervville;">With ANOVA, you can perform this type of analysis with minimal error rates. The F-ratio may be used to compute ANOVA. </span></p><p><span style="font-family: Baskervville;">The F-ratio is a formula for calculating the ratio of mean square error between groups to mean square error within specific groups. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">The procedures for calculating ANOVA are as follows: </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p><span style="font-family: Baskervville;">1. Write the hypotheses and explain why they are needed. A null and alternative hypothesis should be included in every study. </span></p><p><span style="font-family: Baskervville;">2. If the null hypothesis is true, you must assume that the average of the groups is the same. </span></p><p><span style="font-family: Baskervville;">3. The alternative hypotheses' average will be different.</span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><br /></p>Jai Krishna Ponnappanhttp://www.blogger.com/profile/02699917895019277960noreply@blogger.comtag:blogger.com,1999:blog-3702688197392896799.post-82075983082440475832021-06-01T04:56:00.002-07:002021-06-01T04:56:30.827-07:00Data Science's Benefits and Drawbacks <p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTvIrBTpDVDs2nT0r6KV0WoiGvn8cHFP3NjyJT5JkV3ves-ORch1KoXmW7ijP_WzMIRbTN4s3nWXQvgbNgSi828ibDnJQwd0Dma3CfXlnYikYQV9wZUgVFQEGN9u9zlOZhuUBXU-AmLiM/s511/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-12.webp" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="340" data-original-width="511" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTvIrBTpDVDs2nT0r6KV0WoiGvn8cHFP3NjyJT5JkV3ves-ORch1KoXmW7ijP_WzMIRbTN4s3nWXQvgbNgSi828ibDnJQwd0Dma3CfXlnYikYQV9wZUgVFQEGN9u9zlOZhuUBXU-AmLiM/w400-h266/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-12.webp" width="400" /></a></div><br /><span style="font-family: Baskervville;"><br /></span><p></p><p><span style="font-family: Baskervville;">Data science is a rapidly growing profession with several career options. Having said that, there are advantages and disadvantages to this sector. This article examines the benefits and drawbacks of data science in order to assist you in making the best decision possible. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h2 style="text-align: left;"><span style="font-family: Baskervville;">Advantages of Data Science</span></h2><h3 style="text-align: left;"><span style="font-family: Baskervville;"><br /></span></h3><h3 style="text-align: left;"><span style="font-family: Baskervville;">There are several benefits of data science, which are listed in this section. </span></h3><p></p><ol style="text-align: left;"><li><span style="font-family: Baskervville;">Field with the Fastest Growth Data science is a new discipline that is in high demand. </span></li><li><span style="font-family: Baskervville;">Now is the best moment to start your career as a data scientist! a plethora of roles Only a few people possess the abilities required to work as a data scientist. </span></li><li><span style="font-family: Baskervville;">If you want to survive in the profession, you must master a variety of talents and continue to develop. </span></li><li><span style="font-family: Baskervville;">When compared to other machine learning and big data projects, this makes the field less saturated. If you want to work in the field of data science, you have a number of options. The number of data scientists available is quite limited. </span></li></ol><p></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">A Diverse Field </span></b></p><p></p><ul style="text-align: left;"><li><span style="font-family: Baskervville;">Data science may be utilized in a variety of disciplines, although it is most commonly utilized in healthcare, consulting, e-commerce, and finance. </span></li><li><span style="font-family: Baskervville;">Data science is multifaceted, and you may work in a variety of sectors. </span></li></ul><div><span style="font-family: Baskervville;"><br /></span></div><p></p><p><b><span style="font-family: Baskervville;">Makes Data Use Easier </span></b></p><p></p><ul style="text-align: left;"><li><span style="font-family: Baskervville;">Every business need trained workers to gather, process, analyze, and display data. These individuals are data scientists, which means they not only evaluate data but also improve its quality. </span></li><li><span style="font-family: Baskervville;">A data scientist understands how to improve and enhance data so that the organization can make more informed decisions. </span></li></ul><p></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">A Prominent Career </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p></p><ul style="text-align: left;"><li><span style="font-family: Baskervville;">A data scientist enables a business to make the best decisions possible. Many businesses have enlisted the help of data scientists to supply them with the information they need to make well-informed choices. As a result, a data scientist has a significant role inside the company. You may make a lot of money because most organizations are seeking for data scientists. </span></li><li><span style="font-family: Baskervville;">According to Glassdoor, you may make around $160,000 per year. Redundancy should be eliminated. Data science is employed in a variety of sectors, and most algorithms employed in data science assist workers complete less duplicate activities. Most businesses gather historical data, which they may use to train robots to do duplicate activities, therefore simplifying certain human activities. </span></li></ul><p></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">Improve Your Product and Market Intelligence </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p></p><ul style="text-align: left;"><li><span style="font-family: Baskervville;">Data science is a field in which machine learning is used. In machine learning, there are three types of algorithms: supervised, unsupervised, and reinforcement learning. These algorithms look at data sets to identify consumer behavior. </span></li><li><span style="font-family: Baskervville;">Most e-commerce websites, for example, employ recommendation algorithms to give customers with information based on their buy history. As a result, computers are better able to grasp how people behave. </span></li></ul><p></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">Save People's Lives </span></b></p><p><span style="font-family: Baskervville;"><br /></span></p><p></p><ol style="text-align: left;"><li><span style="font-family: Baskervville;">Data science is used in the healthcare industry to enhance diagnostics and patient forecasts. </span></li><li><span style="font-family: Baskervville;">The healthcare industry has discovered a technique to detect tumors and cancer at an early stage using machine learning algorithms. There are several more advantages of employing data science in the healthcare business. </span></li></ol><p></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">Assist with Personal Development </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p></p><ol style="text-align: left;"><li><span style="font-family: Baskervville;">Data science is not only a rewarding career path, but it also allows you to advance professionally and personally. </span></li><li><span style="font-family: Baskervville;">You will acquire the correct mindset and thought process to tackle problems if you want to become a data scientist. Because data science is a blend of management and IT, you will get knowledge from both sectors of business. </span></li></ol><p></p><p><span style="font-family: Baskervville;"><br /></span></p><h2 style="text-align: left;"><span style="font-family: Baskervville;">Drawbacks of Data Science</span></h2><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">Data science is a popular career path, and many individuals pursue it because it pays well. However, there are certain drawbacks to the field.</span></p><p><span style="font-family: Baskervville;">You should also consider the downsides of data science if you want to have a better understanding of it. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">The term "data science" is a bit of a misnomer. </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p><b><span style="font-family: Baskervville;">Data science does not have a clear definition or meaning. It's become a buzzword for analysis, so it's difficult to define what data science is and what a data scientist can do. The job of a data scientist is determined by the company's operations. </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p></p><ol style="text-align: left;"><li><span style="font-family: Baskervville;">Data Science is impossible to master. As previously stated, data science is a synthesis of several disciplines, including computer science, mathematics, and statistics. </span></li><li><span style="font-family: Baskervville;">It is impossible to master the areas employed in data science, which means that you will never be an expert in them. While most online courses have attempted to cover the void that individuals in the data science field are experiencing, this is unachievable. </span></li><li><span style="font-family: Baskervville;">People having a background in statistics may not have all of the requisite computer science knowledge. </span></li><li><span style="font-family: Baskervville;">If you want to stay current in this sector, you'll need to continuously learning new aspects of data science. It necessitates a great deal of domain knowledge. . </span></li><li><span style="font-family: Baskervville;">If you don't have adequate previous knowledge in computer science, statistics, or math, you may find it difficult to address a data science challenge. </span></li><li><span style="font-family: Baskervville;">The same may be stated in the opposite direction. Assume you work for a health-care organization and are responsible for analyzing genetic sequences. You'll need some knowledge of molecular biology and genetics to complete this. This is the only way you'll be able to make informed judgments that will benefit the organization. It will be tough for you to work on evaluating genetic disorders if you do not have this background.</span></li></ol><p></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">Unexpected Outcomes </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p></p><ol style="text-align: left;"><li><span style="font-family: Baskervville;">Data scientists examine the information in the data collection and make educated conclusions based on the patterns and variables found within. This assists you in making well-informed judgments. </span></li><li><span style="font-family: Baskervville;">There are occasions when the data supplied is arbitrary, and you may not get the results you anticipate. </span></li><li><span style="font-family: Baskervville;">The outcomes may also differ owing to inefficient resource usage and data handling. </span></li></ol><p></p><p><span style="font-family: Baskervville;"><br /></span></p><p><b><span style="font-family: Baskervville;">Data scarcity </span></b></p><p><b><span style="font-family: Baskervville;"><br /></span></b></p><p></p><ol style="text-align: left;"><li><span style="font-family: Baskervville;">For many businesses, data is the new oil, and most organizations engage data scientists to analyze the data they acquire and make educated decisions. </span></li><li><span style="font-family: Baskervville;">However, the data utilized in these operations may result in a data breach. </span></li><li><span style="font-family: Baskervville;">Most clients' personal information is maintained by parent firms, and some of these organizations lack adequate protection to avoid data leaks. </span></li><li><span style="font-family: Baskervville;">Many nations have recently developed legislation and recommendations to avoid data breaches and protect personal information.</span></li></ol><p></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><br /></p>Jai Krishna Ponnappanhttp://www.blogger.com/profile/02699917895019277960noreply@blogger.comtag:blogger.com,1999:blog-3702688197392896799.post-83484597932228223812021-06-01T04:43:00.000-07:002021-06-01T04:43:09.577-07:00Data Science Lifecycle - 6 Phases to Reliable Results<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4fxT2ZvVQqN9A24gzTztdbz9i8WqTMF2Svs8oofpktL1vcT952enj1FgFLZeLBa_xRFZlgDNAKgTs6e8TIYU8QcjuIFYV-cJH-7buC2bgtEAyXBnZzq7Y1ppmwNoPH8Ac8iQ4ZsdF3CQ/s510/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-2.webp" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: Baskervville;"><img border="0" data-original-height="340" data-original-width="510" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4fxT2ZvVQqN9A24gzTztdbz9i8WqTMF2Svs8oofpktL1vcT952enj1FgFLZeLBa_xRFZlgDNAKgTs6e8TIYU8QcjuIFYV-cJH-7buC2bgtEAyXBnZzq7Y1ppmwNoPH8Ac8iQ4ZsdF3CQ/w400-h266/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-2.webp" width="400" /></span></a></div><span style="font-family: Baskervville;"><br /></span><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;">Let's take a look at the data science lifecycle. The majority of individuals jump right into utilizing the models they construct on data sets without first learning the fundamentals of data science. </span></p><p><span style="font-family: Baskervville;">Before you go into using the model, you must first grasp these fundamentals and examine the business requirements. </span></p><p><b><span style="font-family: Baskervville;">You guarantee that your results are reliable, make sure to follow the steps of the data science lifecycle. This article provides a high-level summary of the data science lifecycle's phases. </span></b></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">1. Discovery. </span></h3><p><span style="font-family: Baskervville;">Before you begin working on the project, you should be aware of the following: Needs of the business Detailed specifications Budgets are either required or authorized. Priorities are important. You must be able to ask key questions if you want to pursue a career in data science. You must determine whether you have the necessary resources, people, technology, data, and time to support the project's task. This is the stage in which you define the problem and the hypothesis you wish to test. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">2. Preparation of Data</span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">When you've identified the resources you'll need to complete the analysis, you'll need to create or find an analytical sandbox where you can test and analyze the data. Before you model the data, you must analyze, investigate, and condition it. To bring the data into the sandbox environment, you must additionally conduct the following steps: Transform and extract Transform the load To clean, manipulate, and display the data utilized in the research, most data scientists utilize R or Python. These programming languages aid in the detection of data outliers. You may also utilize the data to create or discover a link between variables. After the data has been cleansed and processed, you may use it to do several sorts of analysis. Let's have a look at how you can accomplish this. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">3. Plan the Model </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">Identify the approaches and procedures that will assist you in drawing the link between the various variables in the data set at this step. These connections will aid you in deciding which algorithms to apply in the next step of the lifecycle. To do so, you'll need to use numerous equations and visualization ways to use exploratory data analytics methodologies and technologies. Let's have a look at some of the tools that were utilized for this: : R : This programming language contains a number of modelling features. If you are a newbie, it is also a wonderful platform to utilize to design the proper models. SQL : SQL is a set of strategies for performing database analysis utilizing various prediction models and mining algorithms. ACCESS or SAS: These tools can access data from a variety of storage platforms, such as Hadoop, and utilize it to build a reusable and repeatable model. You may construct modelling approaches using a variety of programs on the market, but R is the most popular. You'll have the necessary insights into your data at the conclusion of this step, which will assist you decide which algorithm to apply. The next step is to put this algorithm to work and construct the model. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">4. Build the Model </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">You must now divide the data set into training and testing data sets after deciding the method to utilize. In this step, you must evaluate the available tools to see if they are enough for the task of creating a model. Make sure you find a stable environment in which to run the models. To create the model, you must examine several strategies like as clustering, classification, and association. To construct the model, you may utilize a variety of tools. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">5. Put the Model to Work </span></h3><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;">You run the data through the model in this step and produce the results and technical papers. You may also need to test the model in the production environment to see if it performs as expected. This will show you how the model works with real-time data. You may also determine the model's limitations. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">6. Disseminate the Information </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">It's critical to assess if the model produced the outcomes you required. This may be accomplished by examining your hypotheses. This is the final step of the data science lifecycle, and it is here that you identify and present the main results to the enterprise. Based on the criteria you established in the first step, you may decide the model's outcomes.</span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><br /></p>Jai Krishna Ponnappanhttp://www.blogger.com/profile/02699917895019277960noreply@blogger.comtag:blogger.com,1999:blog-3702688197392896799.post-78431113991167442922021-06-01T04:34:00.001-07:002021-06-01T04:34:25.953-07:00Who or What is a Data Scientist? <p></p><div class="separator" style="clear: both; text-align: center;"><span style="font-family: Baskervville;"><br /></span></div><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4UhvQItAea59QmO5wOQ8KExawhaJmzTWJcbqj2pN5E5PTvPTwFI0SXv7B7byqFIUer_NGOZphaF0f99sihzJee2j4l3YMIDQBg4gtgy2myyZwN1Xn8e1zfEQ0OA62f2mGwtDXq7sxI7E/s604/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-9.webp" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: Baskervville;"><img border="0" data-original-height="340" data-original-width="604" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4UhvQItAea59QmO5wOQ8KExawhaJmzTWJcbqj2pN5E5PTvPTwFI0SXv7B7byqFIUer_NGOZphaF0f99sihzJee2j4l3YMIDQBg4gtgy2myyZwN1Xn8e1zfEQ0OA62f2mGwtDXq7sxI7E/w400-h225/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-9.webp" width="400" /></span></a></div><span style="font-family: Baskervville;"><br /></span><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;">If you look up the terms "data scientist" on the internet, you'll probably find a lot of different definitions. Data science is used by a data scientist to address various business challenges and challenges. </span></p><p><span style="font-family: Baskervville;">When people understood that a data scientist uses data, different mathematical or statistical functions and operations, and other scientific areas and applications to make sense of the data in a database, the name "data scientist" was coined. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Data Scientists' Responsibilities </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">A data scientist is a person who uses their knowledge of specialized scientific subjects to solve various data challenges. </span></p><p><span style="font-family: Baskervville;">He uses a variety of mathematical, statistical, and computer science components in his work. He doesn't have to be an expert in any of these disciplines. </span></p><p><span style="font-family: Baskervville;">He would, however, employ some technologies and solutions in order to come up with the best answers and reach critical conclusions for the organization's development and progress. </span></p><p><span style="font-family: Baskervville;">When compared to the data accessible in the data set, a data scientist discovers a way to display the data in a useable format. They deal with data that is both organized and unstructured. Let's take a closer look at business intelligence and how it differs from data science. </span></p><p><span style="font-family: Baskervville;">You've probably heard of business intelligence, and most people mix up data science and business intelligence. We'll look at some of the distinctions between the two to help you understand.</span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Disparities: Data Science and Business Intelligence are two terms that are often used interchangeably. </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">Let's have a better understanding of these words before we look at the distinctions between data science and business intelligence. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h4 style="text-align: left;"><span style="font-family: Baskervville;">Business Intelligence:</span></h4><p><span style="font-family: Baskervville;"><br /></span></p><p></p><ol style="text-align: left;"><li><span style="font-family: Baskervville;">An enterprise can gain insight and hindsight in an existing data collection using business intelligence (BI) to explain various trends in the data collection. </span></li><li><span style="font-family: Baskervville;">Businesses may use BI to gather data from both internal and external sources, prepare it, and execute queries on it to get the information they need. </span></li><li><span style="font-family: Baskervville;">They may then develop the necessary dashboards in order to answer various queries or find answers to various business challenges. Businesses can also use BI to assess specific future events. </span></li></ol><p></p><p><span style="font-family: Baskervville;"><br /></span></p><h4 style="text-align: left;"><span style="font-family: Baskervville;">Data science:</span></h4><p><span style="font-family: Baskervville;"><br /></span></p><p></p><ol style="text-align: left;"><li><span style="font-family: Baskervville;">Data science, on the other hand, takes a unique approach to data analysis. You can explain any knowledge or insight in the data set using a forward-looking method. </span></li><li><span style="font-family: Baskervville;">You may use data science to evaluate current or historical data to forecast results. </span></li><li><span style="font-family: Baskervville;">This is one method most businesses try to make well-informed judgments. They may respond to a variety of open-ended queries. </span></li></ol><p></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">The following characteristics distinguish data science from business intelligence:</span></h3><p><span style="font-family: Baskervville;"><br /></span></p><p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhU9O852FrvBH6iXmQjcZXF6sLe4o_6zyP42YrwtEWqcSU5cfdAWHZE068MfGnYZ2lR0_xAkZXURkt4RL0jQeuISv5YLeoTr6CP7o6AT_N6uqVllkgKJlUTGRSnSvUcOpm5EaQ4dngTCTE/s506/image.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><span style="font-family: Baskervville;"><img border="0" data-original-height="204" data-original-width="506" height="161" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhU9O852FrvBH6iXmQjcZXF6sLe4o_6zyP42YrwtEWqcSU5cfdAWHZE068MfGnYZ2lR0_xAkZXURkt4RL0jQeuISv5YLeoTr6CP7o6AT_N6uqVllkgKJlUTGRSnSvUcOpm5EaQ4dngTCTE/w400-h161/image.png" width="400" /></span></a></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><br /></p>Jai Krishna Ponnappanhttp://www.blogger.com/profile/02699917895019277960noreply@blogger.comtag:blogger.com,1999:blog-3702688197392896799.post-73266698001748816432021-06-01T04:26:00.004-07:002021-06-01T04:26:44.808-07:00 Why Should You Use Data Science?<p></p><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi81zsqac4vn-0V9p8y0FiVHrRp5yNWP9dsNm8ptUaIOQwJJDIaGTRlzJoDb2BwUdneMITa3khgjb05aGo56cXgz5SuQJcgCA2meQ5bDi6I9Gb48kviDSUIfoAHVh8pJ8ryDGWHqF6jK2w/s542/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-7.webp" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="340" data-original-width="542" height="251" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi81zsqac4vn-0V9p8y0FiVHrRp5yNWP9dsNm8ptUaIOQwJJDIaGTRlzJoDb2BwUdneMITa3khgjb05aGo56cXgz5SuQJcgCA2meQ5bDi6I9Gb48kviDSUIfoAHVh8pJ8ryDGWHqF6jK2w/w400-h251/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-7.webp" width="400" /></a></div><br /><span style="font-family: Baskervville;"><br /></span><p></p><p><span style="font-family: Baskervville;">Organizations used to deal with limited amounts of data before collecting data from every device they utilized. Using business intelligence tools, it was simple to evaluate and comprehend the facts and relationships within the data set. </span></p><p><b><span style="font-family: Baskervville;">Traditional business intelligence solutions were designed to operate with structured data sets, however today's data is mostly semi-structured or structured. </span></b></p><p><span style="font-family: Baskervville;">It is critical to recognize that the majority of data collected nowadays is semi-structured or unstructured. </span></p><p><b><span style="font-family: Baskervville;">Simple business intelligence systems are incapable of processing this sort of data, especially when enormous amounts of data are acquired from many sources. </span></b></p><p><span style="font-family: Baskervville;">As a result, powerful and complicated analytical techniques and tools are required to process, evaluate, and derive some insights from the data. </span></p><p><span style="font-family: Baskervville;">Data science has grown in popularity for other reasons as well. Let's have a look at how data science is applied in various fields. Service to Customers What a wonderful thing it would be to know exactly what your consumers desire. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Do you believe you can leverage existing data, such as purchase history, browsing history, income, and age, to learn more about your customers? </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p><span style="font-family: Baskervville;">This information may have been available to you in the past. You can efficiently deal with vast quantities of data and discover the proper goods to suggest to your consumers because you employ various mathematical and statistical models. This is a fantastic strategy to increase your company's revenue. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Autonomous Vehicles </span></h3><p><span style="font-family: Baskervville;">How would you feel if you could drive yourself home in your car? Several businesses are aiming to create and enhance self-driving automobile technology. To generate a map of the surrounding area, the automobiles acquire live data from numerous sensors such as lasers, radars, and cameras. This information is used by the car's algorithm to decide whether to accelerate, slow down, park, stop, overtake, and so on. Machine learning algorithms are often used in these methods. </span></p><p><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Predictions</span></h3><p><span style="font-family: Baskervville;">Let's look at how data science can be used to predictive analytics. Take the case of weather forecasting. The algorithms gather and evaluate data from planes, satellites, radars, ships, and other sources. This aids in the creation of the essential models. These models may be used to forecast the occurrence of any natural disaster. You can use this knowledge to take the required precautions to save lives.</span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p><p><span style="font-family: Baskervville;"><br /></span></p>Jai Krishna Ponnappanhttp://www.blogger.com/profile/02699917895019277960noreply@blogger.comtag:blogger.com,1999:blog-3702688197392896799.post-23168120682890996342021-06-01T03:57:00.001-07:002021-06-01T03:57:55.947-07:00What is Data Science?<p><span style="font-family: Baskervville;"> <br /></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibOppxp6g1LB2p5wdoOPKK5LHdH84zTkSDjZBQpq_blXf50SdCeVAumaOIZ1r1t6ZYF5iV8Zma1FWUWbfuF5hNqzAexMT41WZtUB8f44shZrHGa0YmtfeJ9tnNnWfzHi1pyymWY-H7AIc/s509/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-1.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: Baskervville;"><img border="0" data-original-height="339" data-original-width="509" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibOppxp6g1LB2p5wdoOPKK5LHdH84zTkSDjZBQpq_blXf50SdCeVAumaOIZ1r1t6ZYF5iV8Zma1FWUWbfuF5hNqzAexMT41WZtUB8f44shZrHGa0YmtfeJ9tnNnWfzHi1pyymWY-H7AIc/w400-h266/data-science-ai-analytics-big-data-technology-jai-krishna-ponnappan-web-1.jpg" width="400" /></span></a></div><span style="font-family: Baskervville;"><br /></span><p></p><p class="MsoNormal"><span style="font-family: Baskervville;">Data has replaced oil as the new commodity, and every
business, regardless of sector, is seeking for innovative methods to handle and
store massive amounts of data. Until 2010, most businesses found this a
difficult task. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">The goal for each organization was to create a framework or
solution that would allow them to store massive amounts of data. Because Hadoop
and other platforms have made it simpler for enterprises to store vast amounts
of data, they are also focusing on techniques and solutions for processing
data. Data science is the only way to do this. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><b>It's crucial to remember that
data science is the way of the future. It's critical to understand what data
science is, especially if you want to contribute value to your company.</b> </span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Data
Science: An Overview </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p class="MsoNormal"><b><span style="font-family: Baskervville;">Data science is a collection of methods, techniques,
philosophies, and languages used to uncover hidden patterns within a data set's
variables. </span></b></p><p class="MsoNormal"><span style="font-family: Baskervville;">This may prompt you to ask how this differs from the data analysis
that has been done for years. The reason is that previously, we could only
utilize tools and algorithms to describe the variables in a data set; however,
data science makes it simpler to anticipate outcomes. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">A data analyst solely
analyses previous data sets to describe what is happening in the present. </span></p><p class="MsoNormal"><b><span style="font-family: Baskervville;">A
data scientist, on the other hand, merely looks at the data to see if there are
any insights to be gained from it. He also employs complex algorithms to
determine the likelihood of an event occurring. He examines the facts from a
variety of perspectives. </span></b></p><p class="MsoNormal"><span style="font-family: Baskervville;">Data science is utilized to make educated judgments
based on existing data set forecasts. To get this information, you may use a
variety of analytics on the data collection. In the next sections, we'll go
through these in more detail. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Predictive Casual Analytics </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p class="MsoNormal"><span style="font-family: Baskervville;">Predictive causal
analytics is required if you wish to create a model that predicts the
possibilities or consequences of a future event. Assume you work for a credit
firm and lend money to people depending on their credit scores. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">You'll be concerned
about your clients' capacity to pay back the money you've given them. Using
payment history, you may create models to do predictive analysis on the data.
This might assist you in determining whether or not the consumer will pay you
on time.</span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Prescriptive Analytics </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p class="MsoNormal"><span style="font-family: Baskervville;">It's
possible that you'll need to employ a model that can make the necessary
judgments and adjust the parameters based on the data set or inquiry. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">You'll
need to employ prescriptive analytics to do this. This type of analytics is
mainly concerned with giving accurate data so that you can make an informed
decision. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">This form of analytics may also be used to forecast a variety of
related events and actions. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">A self-driving automobile is an example of this
sort of analytics. This is something we've looked at before. You may utilize
the data obtained from the automobiles to run a variety of algorithms and
utilize the findings to make the car smarter. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">This makes it easy for the
automobile to make the appropriate judgments when it comes to turning, slowing
down, speeding up, or determining which way to go. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Artificial Intelligence (AI) </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p class="MsoNormal"><span style="font-family: Baskervville;">Make forecasts Using unstructured, semi-structured, and structured data sets,
you may create predictions using a variety of machine learning methods. Assume
you work for a financial institution and have access to transactional data. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">To
forecast future transactions, you'll need to create a model. You'll need a
supervised machine-learning method to complete this analysis. These methods are
used to train the computer with previously collected data. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">You may also design
and train a model to detect potential frauds based on previous data using
supervised machine learning methods. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><br /></span></p><h3 style="text-align: left;"><span style="font-family: Baskervville;">Pattern Recognition </span></h3><div><span style="font-family: Baskervville;"><br /></span></div><p class="MsoNormal"><span style="font-family: Baskervville;">You won't find
variables in every data set that you can utilize to create the appropriate
predictions. This isn't correct. Every data collection contains a hidden
pattern, which you must discover in order to generate the needed predictions. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">Because there are no pre-defined labels in the data set with which to
categorize the variables, you'll need to utilize an unsupervised model.
Clustering is one of the most frequent techniques for detecting patterns. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">Assume you work for a telephone firm and are entrusted with determining where
towers should be placed in order to construct a network. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;">The clustering
technique may then be used to determine where towers should be placed to
guarantee that every user in the region receives the best signal strength. </span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><b>It's
critical to grasp the differences between data science and data analytics
methodologies, based on the examples above. Only to a limited extent does the
latter encompass the use of forecasts and descriptive analytics. Data science,
on the other hand, is mainly concerned with the use of machine learning and predictive
casual analytics. Now that you know what data science is, let's look at why
companies need to employ it in the first place.</b><o:p></o:p></span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><br /></span></p><p class="MsoNormal"><span style="font-family: Baskervville;"><br /></span></p><p class="MsoNormal"><br /></p>Jai Krishna Ponnappanhttp://www.blogger.com/profile/02699917895019277960noreply@blogger.com