The softmax activation function is commonly used as the output layer in a neural network.
[Read More]
Demystifying gradient descent with code (Go)
Understanding the gradient descent algorithm with Go code.
Gradient Descent is one of the most basic and fundamental algorithms in machine learning. In this post I’ll attempt to explain how the algorithm works with Go code.
[Read More]
Spark DataFrame transform using a User Defined Function (UDF)
Transform a Spark DataFrame or Dataset using a UDF.
This is an extension of my post on Transforming Spark Datasets using Scala transformation functions.
[Read More]
Building an analytical data lake with Apache Spark and Apache Hudi - Part 1
Using Apache Spark and Apache Hudi to build and manage data lakes on DFS and Cloud storage.
Most modern data lakes are built using some sort of distributed file system (DFS) like HDFS or cloud based storage like AWS S3. One of the underlying principles followed is the “write-once-read-many” access model for files. This is great for working with large volumes of data- think hundreds of gigabytes...
[Read More]
Running Docker in Production
Things you should know when running Docker in production.
If you haven’t read about the bare essentials of running Java applications with Docker, you can find it here. In this post we’ll dive deeper into a few advanced concepts that need to be understood when dealing with docker application in a production environment.
[Read More]
Bare essentials of running Java applications with Docker
Using Docker containers to run your Java / JVM Applications
Hopefully you are here because you are convinced that you want to use Docker to run your Java / JVM application. In this post I will go over the most important concepts you need to know in order to run your Java applications within Docker containers in a production environment....
[Read More]
Transforming Spark Datasets using Scala transformation functions
There are few times where I’ve chosen to use Spark as an ETL tool for it’s ease of use when it comes to reading/writing parquet, csv or xml files. Reading any of these file formats is as simple as one line of spark code (after you ensure that you have...
[Read More]