flac
A rust implementation of the audio format FLAC, free lossless audio codec.
The first significant project I started to learn more about the language Rust, was my attempt at implementing a library for the lossless audio format FLAC. The most obvious first goal was to be able to parse any audio file, in that format, that was thrown at this project. The best thing about audio formats that are open to the public is being able to read the specification and start implementing on that without much worry about messing up the finer details.
One of the biggest things I learned working on this project was understand and parsing formats that are binary based, rather than character based. Things like endianness, bit/byte flags, byte padding, and data validation. These skills help me a lot in space efficiency for things like message protocols being sent from server to server and basic compression techniques for data storage.
Parsing was fairly easy because the library I was using nom, allow me to organize my code in a readable fashion. Luckily enough, one of my goals when creating this project was providing enough documentation to follow even the most complex parts of the library without need to look elsewhere. Next up was decoding the audio data into digital waves for each channel. For FLAC there are four levels for compression and the first two -- constant and verbatim -- are the easiest to figure out. Constant is a single value for the length of a block, which represent flat noise like silence. Verbatim is data that couldn't be compressed because there was no clear predictable pattern, essentially random noise.
The last two compression types are both linear predictions, but in a different style, fixed and finite impulse response or FIR. Fixed linear prediction has a set list of polynomials that are use as coefficients, based on the order the data block is stored with -- the range being 0 through 4. FIR linear prediction is similar to fixed, but the coefficients are stored with the data block to figure out what order to used in the prediction -- the range being 0 through 32.
With all these features put together, I was able to fully convert several different FLAC files into uncompressed WAV files that I could play without much issue. The next features would be to encode uncompressed files, but what I completed in this project was enough to understand a lot about binary file parsing, compression algorithms, and data validation.
highlight.js
Javascript syntax highlighter.
As one of my first open source project I contributed to, the skill I picked up was communicating as effectively as I could in writing. Considering we were all in different places in the work, with different time zones, communication was important in getting new features into the master repository.
My introduction was adding Haskell language syntax highlighting into this project. The main this I ended up learn was regular expressions, which have become a very helpful tool for handling text-based tasks like parsing and search. Bringing this language definition into project also granted me the responsibility of maintaining this piece of code. So when bug were reported or requests to improvements, I was the one being pinged to handle them.
The next feature was porting solarized, both light and dark, over to highlight.js as a theme. Pretty simple implementation that only touched CSS files, but these contributions lead me to becoming a core developer. Becoming a core contributor gave me the liberty to make bigger structural changes to the project and one of the pain point in maintaining the project was the build script.
One of the biggest issues with the old build script was it require you to have two separate languages from what the project is written in. The build script was written in Python and compressing JavaScript files needed Java installed to use YUI Compressor. The other issue was how long it took to compress our browser builds when it was a release day. So the goal of the new build script was to build it in Node.js and make it fast no matter the compression type.
Being able to understand Python, I dropped into implementing the new build script and used the new knowledge of regular expressions to develop all features from the old script over to the new one. The most obvious being taking the core of highlight.js and merging it with a language file. The complex part of this script was if the language relied on another language like HTML needing JavaScript and CSS for full language highlighting. The last feature was being able to define package builds that will include languages that fall under said package.
Once I identified all feature I implemented one by one, resulting in the final pull request that started the move to get this integrated into the project. The main thing I learned from getting this merge was, started small and getting other contributors involved as soon as possible is important. With that I was also able to simplify the configuration for the continuous integration server now that we only needed Node.js, in addition to solving the other issues I listed above.