Mon blog-notes à moi que j'ai

Blog personnel d'un sysadmin, tendance hacker

Compilation veille Twitter & RSS #2016-41

La moisson de liens pour la semaine du 10 au 14 octobre 2016. Ils ont, pour la plupart, été publiés sur mon compte Twitter. Les voici rassemblés pour ceux qui les auraient raté.

Bonne lecture

Security & Privacy

TLS nonce-nse
One of the base principles of cryptography is that you can’t just encrypt multiple messages with the same key. At the very least, what will happen is that two messages that have identical plaintext will also have identical ciphertext, which is a dangerous leak. (This is similar to why you can’t encrypt blocks with ECB.)
Q and A with Yawning Angel: sandboxing Tor browser
Here’s an interview I just did with our own Yawning Angel, a longtime Tor developer, about his work on a Linux prototype for a sandbox for the Tor Browser.
Experience and updated recipe for using the Signal app without a mobile phone
In July I wrote how to get the Signal Chrome/Chromium app working without the ability to receive SMS messages (aka without a cell phone). It is time to share some experiences and provide an updated setup.
Utiliser Signal impose la présence de l’espion de Google
La semaine dernière, j’ai vu beaucoup de publicité sur Internet à propos du fait que les serveurs de Signal avaient peu de métadonnées à donner au FBI. Même Snowden recommande Signal et tance Google.
New phone: Samsung Galaxy S III phone with Replicant
Thanks to the Bazaar effort of The Guardian Project, I’ve been offered a phone to test F-Droid and other free software apps for Android. I accepted the offer, and chose a Samsung Galaxy S III phone with Replicant 4.2.2, installed and shipped by Tehnoetic.
I’m using it now as my main phone, and since it uses Android 4.x I’m able to install more modern apps than in my old Galaxy Ace (which remains usable with CyanongenMod 7.2 (Android 2.3.7)).

System Engineering

Reducing the MTTD and MTTR of LinkedIn’s Private Cloud
Nuage (French for « cloud ») is what we call LinkedIn’s internal cloud management portal. It allows LinkedIn developers to quickly create new datastores like Kafka topics, Voldemort stores, and Espresso databases, to name just a few in the LinkedIn data centers. The product consists of an HTTP frontend available to LinkedIn developers and a rest.li backend service, which talks to the underlying systems, such as Kafka.
How to solve anything in VCL, part 2: SOA routing and non-ASCII
In « How to solve anything, part 1, » we discussed Andrew Betts’ clever tips for using Fastly’s Custom Varnish Configuration Language (VCL) to collect data at the edge. In this post, we’ll look at how Nikkei (the Financial Times parent company) uses VCL to deal with a service-oriented architecture as well as write synthetic responses with non-ASCII (i.e., Japanese) characters.
Configuration management - How to start testing your salt formulas
Configuration management tools have recently gained a lot of popularity. At trivago we use SaltStack to automate our infrastructure. As the complexity of configuration files and formulas is increasing, we need a fast, reliable way to test our changes.
In this post we explain how you can start building your own test setup.
How-to: Secure Apache Solr Collections and Access Them Programmatically
Data security is more important than ever before. At the same time, risk is increasing due to the relentlessly growing number of device endpoints, the continual emergence of new types of threats, and the commercialization of cybercrime. And with Apache Hadoop already instrumental for supporting the growth of data volumes that fuel mission-critical enterprise workloads, the necessity to master available security mechanisms is of vital importance to organizations participating in that paradigm shift.
Consul Architecture
I approached Consul recently while looking for a service discovery and configuration automation solution for ProxySQL. My colleague Nik Vyzas wrote a great post on this topic, and I suggest you read it. I wrote this article to share my first impressions of Consul (for whomever it might interest).
AWS IO Performance: What’s Bottlenecking Me Now?
When moving to AWS, it can be difficult to pinpoint where your previously high-functioning system is underperforming. AWS has its own internal rhyme and reason, and anyone moving to AWS must be familiar with how AWS operates. One tool that can help you on this front is AWS CloudWatch, which provides metrics and monitoring on core system performance. The focus of this article is how to configure your IO subsystem in AWS. This requires using Cloudwatch to understand your subsystem’s performance. Are you hitting throughput limits? Are you utilizing all your allocated IOPS? What’s the average queue length?
Helm Charts: making it simple to package and deploy common applications on Kubernetes
There are thousands of people and companies packaging their applications for deployment on Kubernetes. This usually involves crafting a few different Kubernetes resource definitions that configure the application runtime, as well as defining the mechanism that users and other apps leverage to communicate with the application. There are some very common applications that users regularly look for guidance on deploying, such as databases, CI tools, and content management systems. These types of applications are usually not ones that are developed and iterated on by end users, but rather their configuration is customized to fit a specific use case. Once that application is deployed users can link it to their existing systems or leverage their functionality to solve their pain points.
Systemd Or How I learned to stop worrying and love newness
Working in the IT world where things are not yet fully connected, integrated and automated leaves a plenty of opportunities for engineers to get their hands dirty by hard DevOps work. Big Data Engineering is such a field of work where there is a lot of cogs lying around waiting to be integrated. In other words, DevOps and, in this particular case, Linux are inevitable and inseparable parts of Big Data Engineering. This is a story about one of those cogs.
Systemd is a relatively new addition to the Linux ecosystem. It is a system and service manager that is compatible with SysV and LSB init scripts and can work as a drop-in replacement for sysvinit. Systemd’s inception was followed by a huge controversy. The flame war is not yet fully extinguished but systemd has become a part of many Linux distributions and, in my opinion, is here to stay. This post describes my encounter with systemd.
Docker Swarm par l’exemple
Docker Swarm est l’outil proposé par Docker pour assurer la gestion de clusters Docker, le routage, la scalabilité, … Je vous propose quelques exemples mettant en oeuvre Virtualbox, AWS, Azure et OVH.

Monitoring

6 Essential Steps to Reducing Incident Resolution Time
If you find yourself shouting this question at the sky, you’re hardly alone. It’s a chronic support problem. How do you reduce incident resolution time? As it turns out, there are some very effective and very sensible things that you can do. We’ll take a look at them in this post.
Zabbix::Tiny Simple Usage
Zabbix::Tiny is a Perl module that I wrote to automate much of the boilerplate that I found myself writing when I wanted to script against the Zabbix API. This article will take a brief look at exactly what advantages are provided by using Zabbix::Tiny. Finally a simple example script is provided, along with a step-by-step explanation of what it does.

Software Engineering

The Future of Feature Flags: Managing Dynamic Applications
Traditionally, software teams have used feature flags/toggles to control simple rollouts and enable kill switches. Boolean values were used for « on » or « off », « true » or « false ». With the introduction of multivariate flags, developers have been experimenting with serving rich values via these flags: strings, numbers, JSON objects, and JSON arrays. This has opened up a whole new world of dynamic application management.
Managing good bug reports
Bug reporting is an art form that is too often neglected in software projects. Bug reports allow contributors to participate without deep technical knowledge and at the same time provide a crucial space for developers to be made aware of issues with their software that they could not have foreseen or found themselves, for lack of resources, variety or imagination.
Best Practices for Advanced Deployment Patterns
During the episode, we discussed the differences between deployments and releases, as well as advanced deployment patterns and tactics such as rolling, Blue/Green, Canary, Big Bang, feature flags, and dark launches.
Designing Applications for Failure
I recently had the opportunity to attend an AWS bootcamp Herndon, VA office and a short presentation given by their team on Designing for Failure. It opened my eyes to the reality of application design when dealing with failure or even basic exception handling.
Domain Generation Algorithms – Why so effective?
Domain Generation Algorithms(DGAs) are used in malware to generate a large number of domain names that can be used in communications to the malware’s command and control servers. One reason that DGAs are used is because a predefined list of domains that will be used as Command & Control (C&C) servers can be easily discovered in the binaries of malware. An algorithm needs to be reverse engineered. Some DGAs can be completely thwarted through reverse engineering, and every possible domain name can be known and then blocked through security layers. The majority of the domain names generated by DGAs do not resolve (NXdomains) and are never registered with any hosting company by the malware author. They will however, create noise in network logs and annoy Analysts attempting to find active C&C domains.

Databases Engineering

MySQL & MariaDB

MySQL 5.7 Performance Tuning Immediatedly After Installation
This blog updates Stephane Combaudon’s blog on MySQL performance tuning, and covers MySQL 5.7 performance tuning immediately after installation.
A few years ago, Stephane Combaudon wrote a blog post on Ten MySQL performance tuning settings after installation that covers the (now) older versions of MySQL: 5.1, 5.5 and 5.6. In this post, I will look into what to tune in MySQL 5.7 (with a focus on InnoDB).
Encrypt your –defaults-file
This blog post will look how to use encryption to secure your database credentials.
In the recent blog post Use MySQL Shell Securely from Bash, there are some good examples of how you might avoid using a ~/.my.cnf – but you still need to put that password down on disk in the script. MySQL 5.6.6 and later introduced the –login-path option, which is a handy way to store per-connection entries and keep the credentials in an encrypted format. This is a great improvement, but as shown in Get MySQL Passwords in Plain Text from .mylogin.cnf, it is pretty easy to get that information back out.
MySQL 8.0 Labs – Descending Indexes in MySQL
Starting with the 8.0 optimizer labs release the MySQL server now supports descending indexes. As I will detail in this post, this new feature can be used to eliminate the need for sorting results, and lead to performance improvements in a number of queries.
MySQL 8.0 Data Dictionary: Background and Motivation
Just as you use a database like MySQL to store your application data, MySQL must also store its meta data (schema names, table definitions etc) somewhere. Traditionally this meta data storage has been split between many different locations (.FRM, .PAR, .OPT, .TRN and .TRG files). This has gradually become a bottleneck in various contexts.

Elasticsearch

Getting rid of the fantom indexes menace on Elasticsearch zombi masters
Split brains is a recurring problem when running any kind of clusters. A sudden server crash or network partition might lead to inconsistent state and data corruption. Elasticsearch addresses this problem by allowing multiple nodes to be configured as master. Running an odd number of master node and properly setting discovery.zen.minimum_master_nodes to (number of master nodes / 2) + 1 is an easy way to prevents split brain disasters.

Data Engineering & Analytics

Differences between Data Mining and Predictive Analytics
Data mining is an integrated application in the Data Warehouse and describes a systematic process for pattern recognition in large data sets to identify conclusions and relationships. Using statistical methods, or genetic algorithms, data files can be automatically searched for statistical anomalies, patterns or rules.
To Be Continued…
Our objective in improving the Netflix recommendation system is to create a personalized experience that makes it easier for our members to find great content to enjoy. The ultimate goal of our recommendation system is to know the exact perfect show for the member and just start playing it when they open Netflix. While we still have a long way to achieve that goal, there are areas where we can reduce the gap significantly.
Predictive Goes « Geographical » With Dataiku & Esri
Whatever your industry use case and goal, when it comes to predictive, you have to deal with the geographical dimension. Those of us who have already tried know exactly how complex it can be to enrich models with new data based on geographical dimensions: searching and finding trustable data, checking quality, managing different files, merging, cleaning and testing hundreds of tasks is the stuff of a data scientist’s geographic nightmare.

Network Engineering

Building one of the highest-capacity subsea cables in the Pacific
Facebook wants to make it possible for people to have deep connections and shared experiences with the people who matter to them most — anywhere in the world, and at any time. We’re always evaluating new technologies and systems to help us do that, and one of the things we’ve been building out in recent years is our global network infrastructure. Increased capacity, flexible traffic routes, and adaptable system equipment are all things we look for when we consider new projects, which is why we’re partnering with Google and Pacific Light Data Communication (PLDC) to build a new, state-of-the-art subsea cable spanning the Pacific called Pacific Light Cable Network (PLCN).
New undersea cable expands capacity for Google APAC customers and users
Google’s mission is to connect people to the world’s information by providing fast and reliable infrastructure. From data centers to cables under the sea, we’re dedicated to building infrastructure that reaches more people than ever before.
TP Mobilité et réseaux sans fil : réseau sans fil sécurisé et monitoré + mobilité IPv6
Cet article relate la mise en œuvre d’un réseau Wi-Fi sécurisé et monitoré du type de ce que l’on peut trouver dans une société commerciale, une université ou une association. Pour un point d’accès Wi-Fi ouvert à domicile, je recommande plutôt cet excellent tutoriel : Mise en place d’un réseau Wi-Fi ouvert - Emile (iMil) Heitor.
What Can You Do with One Million RIPE Atlas Credits?
What Can You Do with One Million RIPE Atlas Credits? RIPE Atlas lets you make customised measurements from thousands of probes around the world. These measurements cost credits, which users usually earn by hosting or sponsoring RIPE Atlas probes - or you may have just been given a million RIPE Atlas credits as a member of one of the Regional Internet Registries. Find out how you can best spend your credits to check your services’ connectivity and troubleshoot potential issues. https://labs.ripe.net/Members/becha/what-can-you-do-with-one-million-ripe-atlas-credits https://labs.ripe.net/logo.png
Say Cheese: a snapshot of the massive DDoS attacks coming from IoT cameras
Over the last few weeks we’ve seen DDoS attacks hitting our systems that show that attackers have switched to new, large methods of bringing down web applications. They appear to come from the Mirai botnet (and relations) which were responsible for the large attacks against Brian Krebs.
Making Network Performance Monitoring Relevant for Cloud and Digital Operations
This post is based on a webinar presented by Jim Frey, VP Strategic Alliances at Kentik, on Network Performance Management for cloud and digital operations. The webinar looks at how changes in network traffic flows — due to the shift to the Cloud, distributed applications, and digital business in general — have upended the traditional approach to network performance monitoring, and how NPM is evolving to handle these new realities.

Management & Organization

Lessons Learned from Scaling Uber to 2000 Engineers, 1000 Services, and 8000 Git repositories
For a visual of the growth Uber is experiencing take a look at the first few seconds of the above video. It will start in the right place. It’s from an amazing talk given by Matt Ranney, Chief Systems Architect at Uber and Co-founder of Voxer: What I Wish I Had Known Before Scaling Uber to 1000 Services (slides).
6 simple questions for building a successful team
Before you write me off for publishing yet another f#!ing blog post about building a great team, maybe you should read a little further. This one is not another theoretical list of dos and don’ts but what I learned over the year. It’s a story of success and mistakes, things I wanna repeat and things I’ve promised myself I won’t do anymore.