Image for post
Image for post
Python API for Apache Spark

As we know that PySpark is a Python API for Apache Spark where as Apache Spark is an Analytical Processing Engine for large scale powerful distributed data processing and machine learning applications.


Image for post
Image for post
The Power And Performance Of Azure SQL

We know that Azure SQL Database is under the “Intelligent Cloud” business and also the part of the Azure SQL family which is fully committed for the intelligent, scalable, relational database service built for the cloud technologies.

With Azure SQL, we can get the followings:

  1. Rehost: Move on-premises workloads to the cloud
  2. Refactor: Modernise applications for the cloud
  3. Rearchitect: Build new cloud-native SQL database applications
  4. Rebuild: Take advantage of the innovative capabilities of the cloud

Perfect for intermittent usage: Azure SQL Database serverless is best for scenarios where usage is intermittent and unpredictable and we only pay for compute resources we use on a per-second basis, optimising overall…


Image for post
Image for post

Working as Python developer, data analysts or data scientists for any organisation then it is very important for you to know how to play with Dataframes. We understand, we can add a column to a dataframe and update its values to the values returned from a function or other dataframe column’s values as given below -

# pandas library for data manipulation in python
import pandas as pd
# create a dataframe with number values
df = pd.DataFrame({'Num':[5,10,15,17,22,25,28,32,36,40,50,]})
#display values from dataframe
df

Image for post
Image for post

A data lake is conceptual data architecture which is not based on any specific technology. So, the technical implementation can vary technology to technology, which means different types of storage can be utilized, which translates into varying features.


We always try to keep normalization in our database and maintain table relationship for each record as possible. To maintain normalization, we always put our records in more than two tables by making relationship between them which are highly tide up mostly on primary and foreign key relationship.

Image for post
Image for post

Example: In an organization, based on the performance, some employees got the appraisal but some of them did not get any appraisal. Now, system needs to update the salary in employee master only for those employees who got the appraisal.

Note: We can only update 1 table at a time by using Update command. …


If you are working as a SQL Server developer then you will be responsible for the implementation, configuration, maintenance, and performance of critical SQL Server RDBMS systems and most of the times, you have to follow the agile methodology also.

One of the toughest job is adding columns your existing data table inside SQL Server database with some default values without any failures. There are some points which you should keep in mind the following points, in case you are adding a column in the existing table-

  1. If the column is added nullable, then null will be the value used for existing…

Image for post
Image for post

If you are working as Python developer, data analysts or data scientists for any organisation then it is very important for you to know how to play with Lists and get the requested info such as matching indexes or items from them.

In Python, Lists store an ordered collection of items which can be of different types. Each item in a list has an assigned index value. It is important to note that Python is a zero indexed based language. All this means is that the first item in the list is at index 0.

If you want to get all the occurrences and the position of one or more items in a list by using Python then there are many ways but you need a very sufficient way to get the matching items from the list. …


Image for post
Image for post

The Database Engine sits at the core of the SQL Server components which operate as a service on a machine and they are frequently referred to as an instance of SQL Server.

It is very important decision to choose single instance or multiple instance in database designing based on business requirement, environment set up, cost & budgeting, app/DB size etc. factors.


Image for post
Image for post

If you are working as Python developer, data analytics or data scientists for any organisation then it is very important for you to know how to merge two dictionaries.

In Python, Dictionaries are written with curly brackets{}, and they are the combination of keys and values (key: value). The basic use of dictionaries in Python to store data values like a map, which unlike other Data Types that hold only single value as an element. Dictionaries is known as a collection which is unordered, changeable and indexed. …


If you are working as Python developer, data analytics or data scientists for any organisation then it is very important for you to know how to store and access your data such as you need to pull some database tables into csv or txt files for the further analytics process.

Image for post
Image for post
Export all the database tables in the database into multiple csv files in python

If you want to dump each table in CSV format, it does call for a bit of code. Create a python loop to iterate through all the tables and then execute a SELECT query on each of those tables.

Relational databases are the most common storage used for web content, large business storage, and, most relevant, for data platforms. RDMSs will return results with all the same columns, so what you’re doing doesn’t really fit. What I would do is query the metadata table and get a list of all tables, then query each table and append that to the respective file. …

About

Mukesh Singh

BI Specialist || Azure || AWS || GCP — SQL|Python|R|PySpark — Talend, Alteryx, SSIS — PowerBI, SSRS expert at The Smart Cube

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store