Tuesday, February 28, 2023

AWS LocalStack

not provided by AWS, but related to AWS, very useful

LocalStack Getting Started | Docs

"LocalStack is a cloud service emulator that runs in a single container on your laptop or in your CI environment. With LocalStack, you can run your AWS applications or Lambdas entirely on your local machine without connecting to a remote cloud provider!

Whether you are testing complex CDK applications or Terraform configurations, or just beginning to learn about AWS services, LocalStack helps speed up and simplify your testing and development workflow.

LocalStack supports a growing number of AWS services, like AWS Lambda, S3, DynamoDB, Kinesis, SQS, SNS, and more! LocalStack Pro supports additional APIs and advanced features to make your cloud development experience a breeze!

You can find a comprehensive list of supported APIs on our Feature Coverage page.

LocalStack also provides additional features to make your life as a cloud developer easier! Check out LocalStack’s Cloud Developer Tools."

LocalStack @GitHub

Python, Apache License

docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 localstack/localstack

Monday, February 27, 2023

py: PDF => Excel

 How to Convert a PDF to Excel - Strumenta

The code for this article is on GitHub: PDFToExcel

In this article we are going to see how to extract tables trapped into PDF files and put them in Excel files. 

Saturday, February 25, 2023

D3.js visual shuffle

 Fisher–Yates Shuffle

function shuffle(array) {
  var m = array.length, t, i;
  // While there remain elements to shuffle…
  while (m) {
    // Pick a remaining element…
    i = Math.floor(Math.random() * m--);
    // And swap it with the current element.
    t = array[m];
    array[m] = array[i];
    array[i] = t;
  return array;
For more about the Fisher–Yates shuffle, see the Wikipedia article and Jeff Atwood’s post, “The Danger of Naïveté” (2007). The visualizations in this post were built with d3.js and inspired by sort algorithm visualizations in Robert Sedgewick’s Algorithms in C (1998). See as well these visualizations of merge sort and quicksort.
How to randomize (shuffle) a JavaScript array? - Stack Overflow
array.sort(() => Math.random() - 0.5)

Friday, February 24, 2023

AI/ML HW & SW: TensorFlow vs PyTorch

How Nvidia’s CUDA Monopoly In Machine Learning Is Breaking - OpenAI Triton And PyTorch 2.0

"A handful of years ago, the framework ecosystem was quite fragmented, but TensorFlow was the frontrunner. Google looked poised to control the machine learning industry. They had a first movers’ advantage with the most commonly used framework, TensorFlow, and by designing/deploying the only successful AI application-specific accelerator, TPU.


Instead, PyTorch won. Google failed to convert its first mover’s advantage into dominance of the nascent ML industry. Nowadays, Google is somewhat isolated within the machine learning community because of its lack of use of PyTorch and GPUs in favor of its own software stack and hardware. In typical Google fashion, they even have a 2nd framework called Jax that competes directly with TensorFlow."

The State of Machine Learning Frameworks in 2019

Thursday, February 23, 2023

USB-C: it is complicated...

with same name, just different version, USB-C cables have vastly different features... and names!

beware: some versions are "power only", no data transfer

What Is USB-C? USB Specifications and Generations | C2G


USB 3.2 Gen 1 USB 3.0, USB 3.1 Gen 1 SuperSpeed 5 Gbps 2008-2013

USB 3.2 Gen 2 USB 3.1, USB 3.1 Gen 2 SuperSpeed+, SuperSpeed 10 Gbps 10 Gbps 2013

USB 3.2 Gen 2x2 USB 3.2 SuperSpeed 20 Gbps 20 Gbps 2017

USB4 USB 4 Gen 2x2 20 Gbps 2019

USB4 USB 4 Gen 3x2 40 Gbps 2019

USB Type-C and USB 3.1, USB 3.2, USB4 and USB4 V2 Explained

USB-C - Wikipedia

A device with a Type-C connector does not necessarily implement USB, USB Power Delivery, or any Alternate Mode: the Type-C connector is common to several technologies while mandating only a few of them

What Is USB-C? USB Specifications and Generations | C2G

Explaining 4K 60Hz Video Through USB-C Hub | Big Mess o' Wires

Best USB-C cables 2023: Expert reviews and buying advice | PCWorld

The 4 Best USB-C Video Cables for 2022 – Review Geek

Tuesday, February 21, 2023

FinOps, OpenCost, KubeCost: manage cost of k8s and cloud computing

Kubernetes Cost Management with Matt Ray - Software Engineering Daily podcast interview

Episode 550: J.R. Storment and Mike Fuller on Cloud FinOps (Financial Operations) : Software Engineering Radio podcast interview

OpenCost — open source cost monitoring for cloud native environments | OpenCost — open source cost monitoring for cloud native environments

opencost/opencost: Cross-cloud cost allocation models for Kubernetes workloads @GitHub (Go)

OpenCost models give teams visibility into current and historical Kubernetes spend and resource allocation. These models provide cost transparency in Kubernetes environments that support multiple applications, teams, departments, etc.

OpenCost was originally developed and open sourced by Kubecost. This project combines a specification as well as a Golang implementation of these detailed requirements.

Kubecost | Kubernetes cost monitoring and management tool

FinOps Foundation - What is FinOps?

FinOps is a portmanteau of “Finance” and “DevOps”, stressing the communications and collaboration between business and engineering teams.

Matt Ray (@mattray) / Twitter

Monday, February 20, 2023

ML for RealEstate?

after $ billions lost on automated purchasing, hopefully Zillow has an improved tool...

 Zillow Home Value Index Methodology, 2023 Revision: What’s Changed? - Zillow Research

Sunday, February 19, 2023

SQLite + TypeScript + WASM?

sqlite - npm search

node-sqlite3-wasm - npm

node-sqlite3-wasm is a port of SQLite3 to WebAssembly for Node.js with file system access. node-sqlite3-wasm brings SQLite3 to your Node.js environment without recompiling on every target platform. This is especially useful for Electron applications.

974 kB

sql.js uses emscripten to compile SQLite to webassembly (or to javascript code for compatibility with older browsers).

sqlite3 - npm 

Written in modern C++ and tested for memory leaks
Bundles SQLite v3.40.0, or you can build using a local SQLite

3.24 MB

sqlite - npm :

A wrapper library written in Typescript with ZERO dependencies that adds ES6 promises and SQL-based migrations API to sqlite3 (docs).

"The fastest and simplest library for SQLite3 in Node.js."

9.48 MB
A batteries-included, multi-dialect (PostgreSQL, MySQL, CockroachDB, MSSQL, SQLite3, Oracle) query builder for Node.js

easy-to-use and promise-based Node.js ORM tool for Postgres, MySQL, MariaDB, SQLite, DB2, Microsoft SQL Server, and Snowflake. It features solid transaction support, relations, eager and lazy loading, read replication and more.

2.88 MB

TypeORM supports both Active Record and Data Mapper patterns
TypeORM is highly influenced by other ORMs, such as Hibernate, Doctrine and Entity Framework.

19.6 MB

Prisma is a next-generation ORM that consists of these tools:
This can be a REST API, a GraphQL API a gRPC API, or anything else that needs a database.

24.4 MB

Saturday, February 18, 2023

Friday, February 17, 2023

"feature flags" tool: Statsig

Statsig | Product Observability, now in real-time

Democratizing the Tools of Big Tech with Statsig CEO Vijaye Raji: The Complete History and Strategy

interesting story about Facebook engineering, recommended listening!
some highlights:
  • first days on job as an Engineer in Facebook, after 10 years in Microsoft making compilers, he deployed the code used by 600 Million users (!!)
  • a few years later, with a small team of 6 people, made a feature that created $ 1.6 Billion revenue for Facebook (yes, billions!) What a “return on investment”. Right place and right time….
  • apparently Facebook at any moment has thousands of features that are disabled or enabled by configuration, and can even measure usage and value and cost of each of such feature
  • now created a startup that provides “sw features management” tool for cloud and enterprises

Thursday, February 16, 2023

Docker Desktop Extensions

Now that Docker Desktop is main Dockers "product" it is getting better with extensions.
They are relatively simple to add, since DD is an Electron (desktop web) app, 
and as backend just need docker image.

What are Docker Extensions? | Docker Documentation

What are Docker Extensions - YouTube

Overview | Docker Desktop Extensions Documentation

Extensions are packaged as specially formatted Docker images, which our CLI tool helps to build. At the root of the image filesystem is a metadata.json file which describes the content of the extension. It is a fundamental element of a Docker extension.

An extension can contain a UI part and backend parts that run either on the host or in the Desktop virtual machine. For further information, see Architecture.

Extensions are distributed through Docker Hub. However, development of extensions can be done locally without the need to push the extension to Docker Hub

Wednesday, February 15, 2023

Polars: Python + Rust = 10-20 x faster than Pandas

Episode #402 Polars: A Lightning-fast DataFrame for Python [updated audio] - [Talk Python To Me Podcast]

"When you think about processing tabular data in Python, what library comes to mind? Pandas, I'd guess. But there are other libraries out there and Polars is one of the more exciting new ones. It's built in Rust, embraces parallelism, and can be 10-20x faster than Pandas out of the box."

Polars: pola.rs
Apache Arrow: arrow.apache.org
Polars Benchmarks: pola.rs
Coming from Pandas Guide: github.io
Watch this episode on YouTube: youtube.com
Episode transcripts: talkpython.fm

Tuesday, February 14, 2023

Amazon MemoryDB for Redis vs Amazon ElastiCache

Amazon MemoryDB for Redis vs Amazon ElastiCache | LinkedIn

Amazon MemoryDB for Redis is a durable and fully-managed in-memory database combining both database and caching together thus providing microsecond read latency for data access to applications. Since, it is in-memory database, querying would be lightning faster in MemoryDB as compared to ElastiCache. Also it is strong consistency in read replica across multi Availability Zone (AZ) and hence more reliable during failover scenario. It uses distributed transaction log to enable data durability.

Amazon MemoryDB for Redis FAQs – Amazon Web Services

MemoryDB maintains compatibility with open source Redis and supports the same set of Redis data types, parameters, and commands

Monday, February 13, 2023

UI, API and "Edge"

a good podcast

besides typical “UI client” and “API” server there is a new “middle layer” emerging for “server side rendering” that can run on “edge”. There are various solutions already available, some of them based on React (Remix) some framework agnostic etc.

Saturday, February 11, 2023

story: Graphing Calculator

 Shipping Graphing Calculator With Ron Avitzur - CoRecursive Podcast

... We’re building cool stuff. We’re going above and beyond, and we’re excited. But the project encounters reality, shifting priorities, or budgeting constraints, and the work never goes anywhere. It always feels tragic, but then I move on.

But what if I didn’t let a project get canceled? What if I couldn’t accept that? That is what Ron Avitzur’s story is all about. He is the creator of “Graphing Calculator,” and he would not let it be canceled.

Friday, February 10, 2023

book online: Build Your Own Redis with C/C++

 Build Your Own Redis with C/C++ | Build Your Own Redis with C/C++

Learn network programming and data structures by coding from scratch.

The “Build Your Own Redis” Book is Completed | Blog | build-your-own.org

Redis could be considered one of the building blocks of modern computing. There are not many projects that fit the such role and stood the test of time. Here are some examples that meet my criteria of the “building block”: 

NGINX, SQLite, PostgreSQL, Kafka, Linux kernel, etc.

What makes Redis a good target is that it covers two important subjects of software engineering: network programming and data structures.

Summarized in a quote from Feynman: “What I cannot create, I do not understand”.

also available for purchase
Amazon.com: Build Your Own Redis with C/C++: Learn network programming and data structures by building a Redis-like server from scratch with C/C++.: 9798372815469: Smith, James: Books

question: why not in Rust or/end Go?
also, could this run as WASM?

Thursday, February 09, 2023

SQL in JavaScript (web browser)

 AlaSQL/alasql: AlaSQL.js - JavaScript SQL database for browser and Node.js. Handles both traditional relational tables and nested JSON data (NoSQL). Export, store, and import data from localStorage, IndexedDB, or Excel.

AlaSQL - à la SQL ) [ælæ ɛskju:ɛl] - is an open source SQL database for JavaScript with a strong focus on query speed and data source flexibility for both relational data and schemaless data. It works in the web browser, Node.js, and mobile apps.

Wednesday, February 08, 2023

pg-promise for node.js

simpler (and faster?) access to database, instead ORMs like TypeORM or Sequelize, or just pg driver

vitaly-t/pg-promise: PostgreSQL interface for Node.js @GitHub

Built on top of node-postgres, this library adds the following:

  • Automatic connections
  • Automatic transactions
  • Powerful query-formatting engine + query generation
  • Declarative approach to handling query results
  • Global events reporting for central handling
  • Extensive support for external SQL files
  • Support for all promise libraries

Pg, PostgreSQL client - pure javascript & libpq with the same API. It was authored by Brian Carlson on Dec, 2010.
Pg-Promise, PostgreSQL interface for Node.js. It was authored by Vitaly Tomilov on Mar, 2015.
Sequelize, Sequelize is a promise-based Node.js ORM tool for Postgres, MySQL, MariaDB, SQLite, Microsoft SQL Server, Amazon Redshift and Snowflake’s Data Cloud. It features solid transaction support, relations, eager and lazy loading, read replication and more. It was authored on May, 2011.
Typeorm, Data-Mapper ORM for TypeScript, ES7, ES6, ES5. Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, MongoDB databases. It was authored by Umed Khudoiberdiev on Apr, 2016.

Comparing 4 popular NestJS ORMs - LogRocket Blog

Tuesday, February 07, 2023

web tool: Buffalo in GoLang

 gobuffalo/buffalo: Rapid Web Development w/ Go @GitHub

Buffalo helps you to generate a web project that already has everything from front-end (JavaScript, SCSS, etc.) to the back-end (database, routing, etc.) already hooked up and ready to run. From there it provides easy APIs to build your web application quickly in Go.

Monday, February 06, 2023

web3: Kubo IPFS

ipfs/kubo: An IPFS implementation in Go @GitHub

Kubo was the first IPFS implementation and is the most widely used one today. Implementing the Interplanetary Filesystem - the Web3 standard for content-addressing, interoperable with HTTP. Thus powered by IPLD's data models and the libp2p for network communication. Kubo is written in Go.

Sunday, February 05, 2023

node.js generate UUID + NanoID

no need to import uuid module; node.js embedded crypto can do the job

3 efficient ways to generate UUID in Node.js

import crypto from 'node:crypto'; console.log(crypto.randomUUID()); 00a6fa25-df29-4701-9077-557932591766

ai/nanoid: A tiny (130 bytes), secure, URL-friendly, unique string ID generator for JavaScript

A tiny, secure, URL-friendly, unique string ID generator for JavaScript.

“An amazing level of senseless perfectionism, which is simply impossible not to respect.”

  • Small. 130 bytes (minified and gzipped). No dependencies. Size Limit controls the size.
  • Safe. It uses hardware random generator. Can be used in clusters.
  • Short IDs. It uses a larger alphabet than UUID (A-Za-z0-9_-). So ID size was reduced from 36 to 21 symbols.
  • Portable. Nano ID was ported to 20 programming languages.
import { nanoid } from 'nanoid'
model.id = nanoid() //=> "V1StGXR8_Z5jdHi6B-myT"

{Blocks} in GoLang & Rust

 Using {Blocks} in Rust & Go for Fun & Profit

a substitute for "missing" try/catch in Go: { Blocks }

id := 123 firstName := "John" lastName := "Smith" { result, err := db.Exec( `UPDATE usr SET first_name = $2 WHERE id = $1`, id, firstName, ) if result.RowsAffected() = 0 || err != nil { return err } } { result, err := db.Exec( `UPDATE usr SET last_name = $2 WHERE id = $1`, id, lastName, ) if result.RowsAffected() = 0 || err != nil { return err } }

Saturday, February 04, 2023

Kubernetes @ Linode

Kubernetes for Full-Stack Developers, Free Guide | Linode

try with $100 credit.
This credit will be applied to any valid services used during your first 60 days.

After AWS "EKS" and Azure "AKS", there is also "LKS" (and google GKE)

Friday, February 03, 2023

Tesla new "Semi Gigafactory"

will be "the largest building in the world" when completed...

 Tesla Reveals NEW Semi Gigafactory! - YouTube

Generics in Go

 A Comprehensive Guide to Generics in Go | by Steven Ellis | Jan, 2023 | ITNEXT

  • Since version 1.18, the Go language has been extended to allow adding explicitly-defined structural constraints — called type parameters — to function declarations & type declarations.
func F[T any](p T) {}

type M[T any] []

Thursday, February 02, 2023

react-big-calendar component

jquense/react-big-calendar: gcal/outlook like calendar component @GitHub

An events calendar component built for React and made for modern browsers (read: not IE) and uses flexbox over the classic tables-caption approach.

Inspired by Full Calendar.

Wednesday, February 01, 2023

Microsoft "enterprise" training

Enterprise Skills Initiative: Welcome @ Microsoft

likely part of "enterprise subscription" 

"to help you learn the skills you need to be successful with Azure and other Microsoft technologies."

"Sign in by entering your work email address and following the steps to create your learner profile."

AWS is using Coursera for classes... 
Microsoft was using edX before, maybe still is...

no videos, no quizzes, text with images only so far...

eksctl: CLI for AWS EKS


eksctl is a simple CLI tool for creating and managing clusters on EKS - Amazon's managed Kubernetes service for EC2. It is written in Go, uses CloudFormation, was created by Weaveworks and it welcomes contributions from the community. Create a basic cluster in minutes with just one command:

eksctl create cluster

(Go, Apache)

Gale/Udemy classes