Mon blog-notes à moi que j'ai

Blog personnel d'un sysadmin, tendance hacker

Compilation veille Twitter & RSS #2016-24

La moisson de liens pour la semaine du 13 au 17 juin 2016. Ils ont, pour la plupart, été publiés sur mon compte Twitter. Les voici rassemblés pour ceux qui les auraient raté.

Bonne lecture

Security & Privacy

Making Curl | Bash safe(r)
You know those software installation instructions that tell you to download and run a script directly from the internet, as root, using something like the following?
curl https://some-url | bash
Let’s call them « pipe installers ».
Lots of people suggest doing this, including us at Sysdig for our own software.

System Engineering

Greed is Good: Troubleshooting Kubernetes
As Gordon Gekko famously said in the movie Wall Street, « Greed is Good. » In fact you could say the same thing applies in containerized environments – containers want their fair share and then some! (You kinda want to see that Wall Street clip again, don’t you? Here ya go – be on the lookout for a very young Charlie Sheen as well!)
NGINX as a Reverse Proxy for Docker Swarm Clusters
Spawning services across multiple Docker engines is a very cool thing, but those services need to connect each other and be found by public-facing nodes in order to be routed to users. A way to achieve that is to use NGINX as a reverse proxy by defining one or more public-facing nodes. These nodes are going to have NGINX configured to proxy request to each container exposing your service.
Microservices Reference Architecture, Part 3 – The Fabric Model
The NGINX Microservices Reference Architecture is under development. It will be made publicly available later this year, and will be discussed in detail at nginx.conf 2016, September 7–9 in Austin, TX. Early bird discounts are available now.
The Image Optimization Technology that Serves Millions of Requests Per Day
This article will touch upon how Kraken.io built and scaled an image optimization platform which serves millions of requests per day, with the goal of maintaining high performance at all times while keeping costs as low as possible. We present our infrastructure as it is in its current state at the time of writing, and touch upon some of the interesting things we learned in order to get it here.
QUIC, « Make the web faster »
Alors que le web s’éveille doucement à HTTP/2, un autre protocole de communication commence à faire du bruit : Quick UDP Internet Connections. QUIC est un projet développé par Google en parallèle de SPDY. Après l’abandon des recherches sur SPDY (grâce à la sortie de HTTP/2), QUIC va enfin prouver son utilité, en particulier avec les clients mobiles.
Exploiting Zookeeper for managing processes in a production environment with Lockex
As an engineer here at Logentries I need to maintain a complex system that has requirements for being available to our customers. We always build systems with the ability to be resistant to failure.
In our environment, we have processes and daemons which would benefit from having a mechanism for running only one instance at a time. Examples of this might be a producer daemon which can only have one running instance and cron jobs that need to be run at least once from one host in your environment. A more concrete example of the daemon case might be a celery beat process which is responsible for scheduling customer billing reports.

Monitoring

Easy JMX discovery & browsing with the open source Agent
Java Management Extensions (JMX) is a mechanism for managing and monitoring Java applications, system objects, and devices. Most users are familiar with the JMX metrics exposed by applications running in the Java Virtual Machine (JVM) such as Cassandra, Kafka, or ZooKeeper.
JMX represents resources as MBean (Managed Bean) objects. They provide a window through which users can read and interact with the running application.
Optimizing Incident Management for Hybrid Infrastructure
It’s 2016, and your infrastructure is probably hybrid. That means your Incident Management solutions need to be ready for hybrid environments, too. If you only had on-premises servers to manage, and if you didn’t have virtual networks or microservices in the mix, incident management would be much simpler.

Software Engineering

A Git Collaboration Workflow That Provides Feedback Early and Fast
At Aviva Solutions, we’ve been using Git for a little of over two years now and I can wholeheartedly say that after having worked with TFS for years, we’ll never go back… ever. But with any new technology, practice, or methodology, you need to go through several cycles before you find a way that works well for you. After we switched over from TFS, we kept kind of working in a centralized fashion (hey, old habits don’t die easily) where all those feature and team branches are kept on the centralized repository. If the entire codebase involves only a couple of developers, all is fine and dandy. But if you’re working with 20 developers, not so much, unless you love those rainbow-style historical graphs…
API versioning methods, a brief reference
This post lists the most common methods in use to version your API. Its intention is not to convince you to use one or another but rather to provide a comprehensive list of the methodologies used, together with some of their advantages and disadvantages and some additional references for each method as well.
Using the Pipeline Plugin to Accelerate Continuous Delivery – Part 2
In this blog series, we will provide an introduction and step-by-step guide on how to use the Pipeline plugin.
Software Automation On a Budget
When a business is getting off the ground or a startup is launching, it’s understandable that money will be tight and cash flow all but nonexistent. There’s a tangible sense of urgency in getting the business’ core product ready, along with its marketing strategy and other core business functions, rather than focusing on the ideal automation solution.
Anything that isn’t directly related to validating the service offering, winning new customers, or making sales is a legitimate candidate for being put off till some stage in the future. Given that the success of a business isn’t guaranteed, it’s easy to see how thinking long term is an optional extra, even a luxury; something to leave to a later time when the business has obtained some level of certainty and predictability. So many things can fall into this category, one of them being software automation.
Improving CSS quality at Facebook and beyond
With thousands of engineers working across a range of products at Facebook, we face some unique challenges when it comes to code quality.
Not only are we dealing with a large codebase, but things are also moving fast — new features are being added, existing ones are being improved, and things are being reorganized.
For CSS, this means having thousands of files that are in a continuous state of flux.
While we already try to ensure CSS code quality on different levels — through code reviews, style guidelines, and refactoring — unintentional mistakes can also be eliminated with static analysis before they’re even committed.
Until recently, we used a homegrown CSS linter to catch basic errors and ensure consistent style. While it served its purpose well over the years, we wanted a more robust solution.

Databases Engineering

Elasticsearch

Curator 4 is now in beta
It’s been so long since I posted here that people are likely to think I’ve abandoned my blog. It’s partially true. It’s been nearly a year since my last post, and it’s been exactly a year since the post before that.
I figure this is the perfect place to make the announcement for Curator 4. I’ve been very busy with my Logstash work, so Curator 4 is coming out considerably later than I would have liked. Better late than never, right?

Vertica

Jump Start your ETL Application Development with Vertica
Interested in exploring the Vertica Analytic Database in the context of data movement and transformation? To get a feel for it, try our new ETL QuickStart sample apps. You’ll find them on the HPE Big Data Marketplace.
Our Partner Engineering team develops QuickStart apps using tools from our technology partners. Currently we have ETL QuickStarts for the following partner products.

MySQL & MariaDB

InnoDB locks and transaction isolation level
What is the difference between InnoDB locks and transaction isolation level? We’ll discuss it in this post.
Recently I received a question from a user about one of my earlier blog posts. Since it wasn’t sent as a comment, I will answer it here.
Scaling Percona XtraDB Cluster with ProxySQL in Kubernetes
How do you scale Percona XtraDB Cluster with ProxySQL in Kubernetes?
In my previous post I looked how to run Percona XtraDB Cluster in a Docker Swarm orchestration system, and today I want to review how can we do it in the more advanced Kubernetes environment.
There are already some existing posts from Patrick Galbraith (https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/mysql-galera) and Raghavendra Prabhu (https://github.com/ronin13/pxc-kubernetes) on this topic. For this post, I will show how to run as many nodes as I want, see what happens if we add/remove nodes dynamically and handle incoming traffic with ProxySQL (which routes queries to one of working nodes). I also want to see if we can reuse the ReplicationController infrastructure from Kubernetes to scale nodes to a given number.
Scaling Percona XtraDB Cluster with ProxySQL in Docker Swarm
The intention is to be able to start/stop nodes and increase/decrease the cluster size dynamically. This means that we should track running nodes, but also to have an easy way to connect to the cluster.
So there are two components we need: service discovery to register nodes and ProxySQL to handle incoming traffic.
The work with service discovery is already bundled with Percona XtraDB Cluster Docker images, and I have experimental images for ProxySQL https://hub.docker.com/r/perconalab/proxysql/.
MySQL 5.7: New Audit Log Filtering Feature – Part 1
Security auditing plays important role in the process of securing a database system. Thanks to the MySQL Enterprise Audit extension, we can record all activities, such as client connections and execution of queries into a single log file, for later inspection.
Table and tablespace encryption on MariaDB 10.1
MariaDB has a wide set of security features to protect data (see MariaDB Enterprise Security Webinar). To encrypt the data in a MariaDB 10.1 database, you can enable data-at-rest encryption. MariaDB allows the option to select the most suitable level of the encryption in MariaDB: Temporary files, Aria tables, InnoDB tablespaces, InnoDB tables, InnoDB log files and Binlogs. In this article I will explain how to turn on encryption for InnoDB and discuss how encryption affects performance.

Data Engineering & Analytic

It’s Dumb To Not Track All Your Data
According to Ben Porterfield, co-founder and VP Engineering at Looker, there’s no such thing as tracking too much data. In fact, one of the most common mistakes even « smart » people make regarding their analytics is that they simply don’t track enough, he said in his interview with First Round Review.
Untangling Apache Hadoop YARN, Part 4: Fair Scheduler Queue Basics
In Part 3 of this series, you got a quick introduction to Fair Scheduler, one of the scheduler choices in Apache Hadoop YARN (and the one recommended by Cloudera). In Part 4, we will cover most of the queue properties, some examples of their use, as well as their limitations.
10 Steps To Get You Started With Behavioral Analytics
The core of behavioral analytics is events. Events describe any action a user can perform in your application (like opening the app or creating an account, for example) or any activity associated with the user (like push notifications).
Sending optimal event data to your analytics platform is the single most important step toward understanding how your users are engaging with your product. If you’re too hasty in instrumenting your analytics you may never get the full value of your data.

Network Engineering

L’observatoire de la résilience de l’Internet français fête ses 5 ans avec la sortie du nouveau rapport
Mis en place en 2011, l’observatoire de la résilience de l’Internet français vise à améliorer la compréhension collective de ce réseau par l’étude, plus particulièrement, des technologies susceptibles d’entraver son bon fonctionnement et les bonnes pratiques pour y remédier. Il publie chaque année un rapport de synthèse qui fait état de cette étude.
Growing the Wedge/Wedge 100 community
In 2011, we founded the Open Compute Project (OCP), a rapidly growing community of engineers and companies around the world whose mission is to foster more openness, more innovation, and a greater focus on scale in the development of computing technologies. Then, in 2013, we launched OCP’s networking project, and one of our goals was to bring OCP’s mission to the networking industry by disaggregating the hardware and software of networking devices. This allowed us to grow the networking hardware and software ecosystem and enable engineers to build new systems that are more flexible, scalable, and efficient. Over the past few years, we’ve contributed both networking hardware and software to the networking project: Wedge and Wedge 100, our top-of-rack network switches; and FBOSS and OpenBMC, the software libraries that we use on those switches.
Introducing 100 Gigabit Ethernet into the Fastly network
In December we blogged about why speed matters and Fastly’s milestone of serving 1 terabit per second of content to the internet. As we continue to scale our network, we regularly evaluate technology advancements that keep us on the leading edge. One of these advancements is 100 Gigabit Ethernet (GbE) switches in single rack-unit formats, and we’ve recently lit our first 100GbE ports at the Amsterdam Internet Exchange (AMS-IX) using them. There were many questions that had to be answered before we knew that 100GbE was a good fit for our wider environment, and we want to share our experience with the decision-making process.

Management & Organization