Infographic Short Note on Data Pipelines and Data Pipelining in ETL and Data Warehousing

 

Infographic Short Note on Data Modelling and what does it consist of

 

Infographic Note on difference between a Data Engineer vs Software Engineer's Tasks and nature of job

 

Infographic Note on difference between a Data Engineer and a Software Engineer

 

Topics for Python - Basics ( Syntax , Construct , Variables , Operators , Data Structures , Decision Making statements , Loops , Functions , Object Oriented programming )

=====================

Topics for Python - Basics

=====================

 

1)     Syntax

 *   Python is very popular because of its simplicity to use and understand

 *   Simplicity of Python programming language lies in its syntax

 *   Python does not follow any complicated formats for writing of codes

 *  Syntax of Python quite different from other languages like C, C++, Java etc and      much easier to comprehend

 

2)     Variables

 *  If one is familiar over the manner in which variables can be declared over any Python program then the topic of manner and efficient declaration of variables can also be done efficiently

 

3)     Operators

 *   Operators are the symbols that represent a particular action or process .

*  One can explore different mathematical operators and calculations that can happen using the available operators in Python

 

4)     Data Structures

 *   Data Structures are the most important concept of any programming language

 *  Data Structures are the building blocks of any programming language as all             programming languages deal with objects creation which deal with memory      allocation , memory deallocation and memory utilisation of the variables / objects using the relevant data structures for carrying out the necessary operations that are needed to be performed as per the program that one is working upon

 *  Python has especially these four data structures that are lists , sets , dictionaries and tuples .

 *   One can get to have a good understanding of the manner in which each of the above listed data structures are put to use in advanced examples which would discuss the manner in which append , deletion or structural operations take over a given data structure .

 *  The most important concepts of Python that need to be used and understood are the following :

 

1)     Decision Making Statements :

 

If , If-else , Nested If else , chained conditionals of the else if type of statements , single statement conditions all come under the category of decision-making statements . Along with earning , one may also have to practice 10+ coding problems on a daily basis which would help someone get familiar with the  manner in which one can scale up on the level of complexity of the programs and their creation

 

2)     Loops :

 

Loops are the structures that repeat a sequence of instructions . Python deals with while loops , for loops and nested loops . Along with these one may also learn loop control statements which show the manner in which loops can be initialised , implemented , interjected and put to end in the desired way of the programmer .

 

3)     Functions in Python

 Functions are the smaller pieces of code that perform some desired task . Python has four different types of functions that are User Defined functions , Built-In functions , Lambda functions and Recursion functions mainly .

 

We will just put a small recap over Functions in Python :

 

User Defined functions are those functions which a user  / programmer writes himself which takes programming constructs and statements to write a function which would achieve a desired result at the end of the function which may include the use of some built-in functions into the overall function code construct . 

Built-In functions are those functions which are created as a part of the programme package at the time of creation of the Program package and its default libraries which come together with the program at the beginning of the program package .

Lambda functions are also some form of user defined function , however the construct is limited to only one line and this usually does the work of iteration over the elements of the function , access the individual elements , or use the functions to iteratively append , retrieve or delete some elements over an existing structure or a dummy/temporary structure .

  

*  Along with the the basic concepts of the scripting part of the Python Language one also needs to have a good understanding of implementation as well as creation of features in Object Oriented Language features of Python :

 

1)     Polymorphism

2)     Inheritance

3)     Encapsulation

4)     Abstraction

 

All these aspects of Programming in Python and how the same are going to  be used in Python for Data Science and Machine Learning Modelling would be

discussed in further blogs . As I am getting disarrayed due to a some intermittent problems over my laptop I am not able to implement some of the important Algorithms and Modellling Processes . But soon , I hope to get these started on a better level .

 

Last modified: 21:32

Lists and Dictionaries in Python Dictionaries in Python

 When should one use dictionaries and when should one use lists in Python

 =====================================================

 *  Lists are the most widely used data when the data is stored in the form of integral indexes with some starting position to some given position within the Python list

 *  Lists are also mostly used when the data that needs to be maintained in a certain specific manner or  where the data needs to be sorted where finding  some data is only good enough when the user of the given set of code is knowing the exact index or position of the item element where the list item needs to be searched or retrieved

 *  Lists are also used when the ordering of items within the List item object is irrelevant as any given element within a list can be searched and retrieved just by using some conditional loops which will iterate over the list item object and then the item could be put to use directly without any form of replacement or elimination within the data structure

 *  One of the main highlighting important points about Lists as a data structure for item storage is that Lists can be used for storage of other lists and tuples as a data storage within it because of which the main list containing the sub-lists within it could act as a parent list object which can keep within it other child objects for storage and retrieval as well

 *   Now coming to dictionaries and what they are used for :-

 *  Dictionaries are perfect storage objects when the data is to be stored on the basis of positonal indexes coupled with an adjoining value associated for that index position . Here also the ordering of the data is not of great matter and does not need to sort the stored data . Finding any associated data of the dictionary object needs only a key value of that object 

*   Example where a List is used for data storage is an example of a shopping cart where any stored item object's storage does not matter .

*  Example where a Dictionary Item object is used as a storage object is that of a an examination roll / rank number sheet ( parikhya patra ) where people are arranged on the basis of their marks obtained in some examination from Rank 1 to the last rank in the exam . Usually the person who obtains the highest marks in a given exam is attributted the 1st rank and the first roll number for that examination is also alloted to the first rank holder after the results are out , the successive role numbers are also attributed according the net marks obtained  by an examiner in the considered case of the examination till the last rank/roll number of the examiner .

 

*  A dictionary is very useful data storage object where one may need to access a given data based on the item that is present over a given index position within the storage object of the data structure .

 *  As a measure of how important the dictionaries are - every module , class , function and method written in Python in the form of dictionaries where all the attribute names are in the form of Keys and all the attribute values there in can be found quickly .

 *  Therefore when one wants to decide upon which data structure to be used for a certain type of data object one might need to consider the best use cases of both these data structure objects : Lists and Dictionary Items .

 ======================================

To sum up once again for Lists and Dictionaries

 =====================================

 

*  Lists are ordered data structure objects that means that the program has control over the order in which the data is stored . This data is also sortable that means that the program can sort the data into whatever order when the data is needed to be fetched and retrieved , the list is also containing a list of iterable items i,e items that can be accessed one item at a time when the list object is iterated or executed within a loop .

 

*  Dictionaries are like the partially ordered data structures that can control the order or the manner in which the data items are inserted and stored within the object in the form of keys and attributes . Dictionaries are non-sortable and iterable in pairs of both keys and associated value objects .

 

 

 

 

 

 

 

 

 

Last modified: 18:49

Why is Artificial Intelligence a big fab hype in the recent IT and computing technology world - a collective article on current standpoint on AI

=====================================================

Why is Artificial Intelligence  a big fab hype in the IT and technology world

=====================================================

* The most comprehensible perception about artificial Intelligence in the current technological biome is a distopian future coupled with human-like looking and performing robots and realistic looking holograms that throw a completely realistic idea of a certain picture or scenario in which non-human like androids and droids would be acting and talking like real people and having human-  level or even superhuman intelligence and capablities for performing a task which would be much complex and strenuous to humans . This concept is still beyond the aegis of reality as of now and this term is called as "Artificial General Intelligence(AGI)" and this does not exist anywhere on earth till now as depicted by some high-level CGI based movies in their content .

 

* But what is actually existing as of now is that of a fractional level implementation of Deep Learning which is shortly known as DL which can do some small level specific tasks which are better than people doing the computing task .. one   needs to keep in mind over here is that it is only restricted to computing activity and factory work . Still is believed that there are multiple fundamental level limitations that does not properly get developed into AGI - Artificial General Intelligence (AGI) . So the next level of algorithmic data science and machine learning development is that of innovating the present technology to come up with better networks and better methods for shaping into an artificial Intelligence product .

 

* So inorder to get a bird's eye view of where the development stands at , one might need to look at the following persisting scenarios :

 1)     Where Deep Learning and Reinforcement Learning are standing today

 2)    What are the limitations of Deep Learning and Reinforcement Learning algorithms and  where does the current trend stand at ?

 3)    Underlying principles and applications of Human Neuroscience and human intelligence

4)     A possible architecture to achieve artificial general Intelligence

 

Last modified: 14:46