I am a Tech Leader, Engineer and Entrepreneur based in Portland, OR. I am the CTO and Co-Founder at Fiveonefour, where we build a Moose, an open source developper framework for the data and analytics stack.
Thoughts on engineering, leadership, and technology
This blog post explores the structure of the Internet and how it is connected through switches. We introduce the concept of cross bar networks, which are a basic way to connect two arrays of devices to each other. However, building such a network can be expensive when the number of inputs and outputs gets too big. We discuss Clos networks, which are based on cross bar networks but can handle a larger number of inputs and outputs.
This blog post explains matrix multiplication and dot products of matrices. It then discusses how to perform distributed matrix multiplication, which is useful in machine learning applications where large matrices may not fit into memory. The post explains how to segment the data into slices or submatrices, which can be assigned to different processors. The processors compute the dot product for each submatrix, and the results are then combined to produce the final output. The post provides detailed explanations and illustrations for each step of the process.
Introduces the concept of "parallel prefix" computing using an illustrated story of an Italian chef participating in a pizza competition. In this competition, the chef has to make pizzas that represent all the steps of the cooking process, and he realizes he can get other cooks to make the pizzas in parallel. However, the cooks have to communicate and cooperate to overcome the constraint of having only one ingredient on their table at a time. This story is used to explain the concept of parallel prefix, where each cook is a processing unit, and the limit in the ingredient space is the memory limit on each processing unit.