If you are working as a Python developer, data scientist or data analytics then you have to do many task inside the pandas Dataframe such as find the maximum value of each column or find the maximum value of each row.

Pandas Library - As we know that Python comes with a very strong libraries and Pandas is also one of them, which is -
* an open-source,
* BSD-licensed Python library providing high-performance,
* easy-to-use data structures and
* data analysis tools for the Python programming language
Python with Pandas is used in a wide range of fields including…

Sending Email in SQL Server

As we know that SQL language is specified as an ANSI and ISO standard. Performance, scalability, and optimization are very important features for database-driven applications, especially on the Web. In application development, we are required to build a SQL program to perform certain task(s) on periodic basis or daily basis.

For example — send an email alert with Department Wise Employee List from SQL Server to business users. Let us discuss how we can send an email in HTML table format using TSQL. Before sending emails from SQL Server, we should have the following things-

  1. Email Profiler
  2. An activated and…

If you are working as a data engineer where you have to perform many ETLs kind of stuffs such as read the data from csv or txt files and store it into BigQuery tables.

For data analysis and exploration, Jupyter/IPython notebooks has often been the tool of choice for its ease in sharing work and explaining the thought process.
Pandas dataframe to Google BigQuery table

As you know that pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

Prerequisite: Have Google Cloud Platform project already set up

Python is an open sourced, interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together.

Python is simple and straightforward enough where we can become productive pretty rapidly. Now a lot of libraries have been written for Python, making it hard to change to a better language. it’s easy to learn, and currently it has the best AI/Statistics tools

Sometimes we think of python as…

We know that with the right technology, we can do much better than just keep up and if we could also ensure flexible development and make it easier to protect our data, process to access, process and analyze data whenever it’s required. With the right tools and best practices, an organization can use all its data, making it accessible to more users and fueling better business decisions.

New technologies innovations can improve improve the modern cloud-based data lakes, data warehousing and analytics with regard to availability, simplicity, cost, and performance which should be meet current and future needs by ableing…

As a data engineer or python developer and you have to read the data from an excel data file having multiple sheets or tabs and save the data into other data sources like SQL Server database or SQLite then it can be easily possible in Python.

Read Multiple Excel Sheets or Tabs

There is no limitation of rows in csv or text file format but in case of excel file, there are only 1000000 rows allowed in per excel sheet or tab.

In the below Python code, we are using SQLite to store the data from an excel data file having multiple sheets or tabs. As…

A data lake can store all data, regardless of source, regardless of structure, and usually regardless of size also. An Ideal data lake also supports embrace these nontraditional data types which come from nontraditional data sources. These nontraditional data sources include items such as web server logs, sensor data, social network activity, text, and images. New use cases for these data types continue to be identified.

Data Reliability Challenges in Building with Data Lakes

In the data lake, since all data is stored in its raw form, access could be provided to someone who needs to analyze the data quickly. For data science, data lakes provide a convenient…

As we know that now a days businesses are going through unprecedented change and disruption and data is the most important business asset for these organisations which must be audited and protected because it plays a key role in helping organisations make better decisions to get the meaningful insights in their business. While data teams are generally growing, business demand for information and insights continues to surge much faster then Data engineering is the key to unlocking all other data-driven workflows.

Data Engineering Drivers in Building Reliable Data Lakes

To your business growth, you should always focus to increase your customers and always follow a good strategic choice…

Python API for Apache Spark

As we know that PySpark is a Python API for Apache Spark where as Apache Spark is an Analytical Processing Engine for large scale powerful distributed data processing and machine learning applications.

The Power And Performance Of Azure SQL

We know that Azure SQL Database is under the “Intelligent Cloud” business and also the part of the Azure SQL family which is fully committed for the intelligent, scalable, relational database service built for the cloud technologies.

With Azure SQL, we can get the followings:

  1. Rehost: Move on-premises workloads to the cloud
  2. Refactor: Modernise applications for the cloud
  3. Rearchitect: Build new cloud-native SQL database applications
  4. Rebuild: Take advantage of the innovative capabilities of the cloud

Perfect for intermittent usage: Azure SQL Database serverless is best for scenarios where usage is intermittent and unpredictable and we only pay for compute resources…

Mukesh Singh

BI Specialist || Azure || AWS || GCP — SQL|Python|R|PySpark — Talend, Alteryx, SSIS — PowerBI, SSRS expert at The Smart Cube

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store