Showing posts with label data engineering. Show all posts
Showing posts with label data engineering. Show all posts

Sunday, July 25, 2021

Regular Expressions in Python | Scenario - Finding a sub-string match and its count from a main string using Regular Expressions and findall() function in Python | Conceptual Infographic Note


 

Regular Expressions in Python | Scenario - Replacing a word ("sub-string") with another word ("sub-string") in a main string using Regular Expressions and substitute function in Python | Conceptual Infographic Note


 

Regular Expressions in Python | Scenario - Finding matching pattern of the main string using Regular Expressions and findall() function in Python | Conceptual Infographic Note

 


Regular Expressions in Python | Scenario - Finding Positional Indexes of each matching pattern of the main string using Regular Expressions | Conceptual Infographic Note

 


Regular Expressions in Python | Scenario - Finding and Retrieving specific sub-strings with commonality in patterns using Regular Expressions | Conceptual Infographic Note


 

Regular Expressions in Python | Scenario - Finding and Retrieving specific sub-strings with commonality in patterns using Regular Expressions | Conceptual Infographic Note

 


Regular Expressions in Python | Scenario - Finding and Retrieving specific sub-string and their cpount of occurrence from a main parent string using regular expressions | Conceptual Infographic Note


 

Regular Expressions in Python | Scenario - Finding and Retrieving specific positional indexes of sub-strings and index positions from a parent string using regular expressions and store them in one dictionary | Conceptual Inforgraphic Note

 


Regular Expressions in Python | Scenario - Finding and Retrieving specific sub-strings and integers from a parent string using regular expressions and store them in one dictionary | Conceptual Inforgraphic Note


 

Regular Expressions in Python usecase Scenario | Finding and Retrieving specific sub-strings from a parent string using regular expressions and storing in separate list | Conceptual Inforgraphic Note

 


Friday, July 23, 2021

Features of NoSQL Databases and their Usage for Analytics | 5 Conceptual Questions and Answers | Concept Infographic Note


 

Topics discussed in the post the following questionnaire :

Q1) What are some predominant features of No-SQL databases ?

Q2) Why NoSQL is called "SQL with a No " with "No" as the preceding string on SQL ?

Q3) How do organisations leverage NoSQL data for their businesses ?

Q4) Where are NoSQL databases useful ?

Q5) What do you mean by the statement , "The constraints of a relational database are relaxed ? "

Difference between RDBMS and NoSQL databases | 10 Conceptual Questions and Answers | Concept Infographic Note




Questions and answers discussed in the post:
Q1) What are the different ways in which RDBMS databases differ from NOSQL databases?
Q2) What are the Applications Level feature supported by RDBMS and NoSQL Databases ?
Q3) What are the principles of RDBMS and NoSQL type databases ?
Q4) Explain about Availability aspect of RDBMS and NoSQL type databases ?
Q5) What is the data velocity supported over RDBMS and NoSQL databases ?
Q6) What do you mean by Latency . What happens if Latency is High ? And what happens if Latency is Low ?
Q7) What is the data volume supported over RDBMS and NoSQL databases ?
Q8) What are the data sources present over RDBMS and NoSQL databases ?
Q9) What are the different data types that are present over RDBMS and NoSQL ?
Q10) What are the different data accesses that are present over RDBMS and NoSQL ?
Q11) What are the different technologies that are supported over RDBMS and NoSQL ?
Q12) Write about the cost aspects of RDBMS and NoSQL





Thursday, July 22, 2021

Shopping Cart Total Bill Generation Project in Python using Jupyter Notebook | Sample Mini-Project Implementation Infographic Note


 

Concept of "NoSQL Databases" in Big Data Analytics with 7 sample conceptual questions and answers | Infographic Note with explanation

 


Examples of Using "JOIN" function on Python Data Structures | Sample Conceptual Questions with Answers with examples | Infographic Note

 


Example - 01 : Joining Numeric List Elements with char string

Example - 02 : Joining Tuples with character string

Example - 03 : Joining List elements with long string

Example - 04 : Joining Character List Elements with long string

Example - 05 : Joining Character List Elements with null string


Monday, June 7, 2021

Lists and Dictionaries in Python Dictionaries in Python

 When should one use dictionaries and when should one use lists in Python

 =====================================================

 *  Lists are the most widely used data when the data is stored in the form of integral indexes with some starting position to some given position within the Python list

 *  Lists are also mostly used when the data that needs to be maintained in a certain specific manner or  where the data needs to be sorted where finding  some data is only good enough when the user of the given set of code is knowing the exact index or position of the item element where the list item needs to be searched or retrieved

 *  Lists are also used when the ordering of items within the List item object is irrelevant as any given element within a list can be searched and retrieved just by using some conditional loops which will iterate over the list item object and then the item could be put to use directly without any form of replacement or elimination within the data structure

 *  One of the main highlighting important points about Lists as a data structure for item storage is that Lists can be used for storage of other lists and tuples as a data storage within it because of which the main list containing the sub-lists within it could act as a parent list object which can keep within it other child objects for storage and retrieval as well

 *   Now coming to dictionaries and what they are used for :-

 *  Dictionaries are perfect storage objects when the data is to be stored on the basis of positonal indexes coupled with an adjoining value associated for that index position . Here also the ordering of the data is not of great matter and does not need to sort the stored data . Finding any associated data of the dictionary object needs only a key value of that object 


*   Example where a List is used for data storage is an example of a shopping cart where any stored item object's storage does not matter .


*  Example where a Dictionary Item object is used as a storage object is that of a an examination roll / rank number sheet ( parikhya patra ) where people are arranged on the basis of their marks obtained in some examination from Rank 1 to the last rank in the exam . Usually the person who obtains the highest marks in a given exam is attributted the 1st rank and the first roll number for that examination is also alloted to the first rank holder after the results are out , the successive role numbers are also attributed according the net marks obtained  by an examiner in the considered case of the examination till the last rank/roll number of the examiner .

 

*  A dictionary is very useful data storage object where one may need to access a given data based on the item that is present over a given index position within the storage object of the data structure .

 *  As a measure of how important the dictionaries are - every module , class , function and method written in Python in the form of dictionaries where all the attribute names are in the form of Keys and all the attribute values there in can be found quickly .

 *  Therefore when one wants to decide upon which data structure to be used for a certain type of data object one might need to consider the best use cases of both these data structure objects : Lists and Dictionary Items .


 ======================================

To sum up once again for Lists and Dictionaries

 =====================================

 

*  Lists are ordered data structure objects that means that the program has control over the order in which the data is stored . This data is also sortable that means that the program can sort the data into whatever order when the data is needed to be fetched and retrieved , the list is also containing a list of iterable items i,e items that can be accessed one item at a time when the list object is iterated or executed within a loop .

 

*  Dictionaries are like the partially ordered data structures that can control the order or the manner in which the data items are inserted and stored within the object in the form of keys and attributes . Dictionaries are non-sortable and iterable in pairs of both keys and associated value objects .



 

 

 

 

 

 

 

 

 

Last modified: 18:49

Technical Data Analyst posted at HP @Bangalore on 07-06-21 - Self Analysis , Upscaling and Recommendation

Technical Data Analyst posted at HP @Bangalore on 07-06-21

=======================================================================

 *   This job position is posted for Bangalore


 *  The relevant years of experience needed for this job is around 0 to 2 years of experience

 

*  The secondary requirement for the fitting criteria is that the person should have a  Master's degree in computer science or Information systems

 

*   A candidate for the job should be able to leverage and scale open source tools .This aspect is currently I am also not sure and I am also not yet sure which  open source tools are being referred to in this , since collaboration tools are the most requisite open source tools that are common to all the people albeit their technological skills

 *   Proficient is SQL and architecting relational databases - NoSql is a plus{ SQL querying skills is a standard requirement across most of the job positions but the niche plays when it comes to the manner in which one is able to perform some of the SQL operations which comes with experience and exposure . Even I can say that , though I am good .. means good in SQL and would rate myself a mid-level person in expertise over SQL , I am not sure I would be able to handle more difficult or expert level queries without starting over from such minimum requirements .Also my work over NoSQL till now has been limited and has not been much diverse because of which NoSql and Big Data system level requirements is still a new thing for me }

 

*  Proficiency in Public Cloud Providers like Amazon AWS , Google GCP , Microsoft Azure etc is also a must { I have some experience over AWS cloud and am aware of the manner in which some of the applications work over AWS but I am yet to explore all the greater details and aspects of AWS and how big scale projects are deployed at AWS }

 

*     Experience over Industry - standard BI platforms { this necessitates a candidate to have some good amount of experience over working over Business Intelligence and Reporting Applications like Microsoft Power Bi and Tableau where one might be able to carry out and find out report generation and deployment activities in order to find out the necessary actionable items required for the specific work }

 

*   Knowledge of SAAS products and their multitenant cloud implementation { I this area I am just aware of the fundamental theorems that are under work and implementation is still yet to be done and scaled up , so this is one area that I shall again need to brush up and get my hands upon. I shall try to learn from the available online resources and try online cloud platforms over where such projects and modules are deployed}

 ===================================

 Description and Responsibilities on the job

 ===================================

 *  Analyse the business problems to analytics and there upon recommending the most appropriate methods to yield insights and results

 

*  Analyse Sales and Financial Data with Cloud and Operational data to present over to the clients to view

 

*  Data Warehousing and Reporting solutions to address the growing needs for reporting , analytics , and data requirements


 *  To automate cloud metrics to analyse the business requirements and business environments and on top of that gather business requirements and create data visualizations taking into consideration PaaS , Geographical data etc .

 

 

Last modified: 13:31