3 days agoMember-onlyETL — High Quality Data PipelinesAs you are probably aware, data pipeline is a rather broad term. It’s a collection of tasks that transfer, alter, or provide data. A simple example is loading a.txt file into a table/file. It could also be as complex as real-time data aggregations with machine learning scoring and more. The…Data Science4 min readData Science4 min read
Feb 17, 2021Member-onlyIdeal Cloud-based Data Lake FrameworkWe know that with the right technology, we can do much better than just keep up and if we could also ensure flexible development and make it easier to protect our data, process to access, process and analyze data whenever it’s required. …Data Science6 min readData Science6 min read
Feb 13, 2021Member-onlyPython — How to Read Multiple Excel Sheets or TabsAs a data engineer or python developer and you have to read the data from an excel data file having multiple sheets or tabs and save the data into other data sources like SQL Server database or SQLite then it can be easily possible in Python. There is no limitation…Data Science4 min readData Science4 min read
Feb 5, 2021Member-onlyData Reliability Challenges in Building with Data LakesA data lake can store all data, regardless of source, regardless of structure, and usually regardless of size also. An Ideal data lake also supports embrace these nontraditional data types which come from nontraditional data sources. These nontraditional data sources include items such as web server logs, sensor data, social…Data Science5 min readData Science5 min read
Feb 2, 2021Member-onlyData Engineering Drivers in Building Reliable Data LakesAs we know that now a days businesses are going through unprecedented change and disruption and data is the most important business asset for these organisations which must be audited and protected because it plays a key role in helping organisations make better decisions to get the meaningful insights in…Data Science4 min readData Science4 min read
Jan 15, 2021Member-onlyPySpark — Read CSV file into DataframeAs we know that PySpark is a Python API for Apache Spark where as Apache Spark is an Analytical Processing Engine for large scale powerful distributed data processing and machine learning applications.Data Science4 min readData Science4 min read
Jan 7, 2021Member-onlyWhy does Microsoft Azure lead the market?We know that Azure SQL Database is under the “Intelligent Cloud” business and also the part of the Azure SQL family which is fully committed for the intelligent, scalable, relational database service built for the cloud technologies. With Azure SQL, we can get the followings: Rehost: Move on-premises workloads to…Data Science4 min readData Science4 min read
Nov 19, 2020Member-onlyPython — How to update multiple columns at once in dataframe?Working as Python developer, data analysts or data scientists for any organisation then it is very important for you to know how to play with Dataframes. …Data Science3 min readData Science3 min read
Oct 28, 2020Member-onlyWhy does cloud make Data Lakes better?A data lake is conceptual data architecture which is not based on any specific technology. So, the technical implementation can vary technology to technology, which means different types of storage can be utilized, which translates into varying features.Data Science4 min readData Science4 min read
Oct 27, 2020Member-onlySQL Server— Update table using Inner join in SQLWe always try to keep normalization in our database and maintain table relationship for each record as possible. To maintain normalization, we always put our records in more than two tables by making relationship between them which are highly tide up mostly on primary and foreign key relationship. Example: In…Sql5 min readSql5 min read
