obsessive facts blog

[TAGGED: encryption]
  1. The JavaScript Black Hole

    A playbook for ethical engineering on the web.

    In the 25 years since JavaScript was first added to Netscape Navigator, the language has evolved from a cute little toy to an integral part of the Internet. JavaScript frameworks such as React and Angular have transformed the web, bringing us fully-fledged client side applications with functionality that could only be imagined just a decade ago. In the process, the web has become more powerful, but also much more dangerous. Malware and mass surveillance have become persistent threats, fueled by the ever-expanding amounts of user data exposed by new JavaScript features, and sucked into the black hole of omnipresent tracking networks. With real human costs, these threats have been worsened by the increasingly popular belief that "the web browser is an operating system, and everything is an app."

    This essay is written for web developers and people interested in the field. In it, I break down the problems mentioned above, demonstrate some commonly-used JavaScript practices that can expose users to harm, provide examples of actual harm being done, and ultimately propose some actionable alternatives that we, as developers, can adopt to prioritize ethical engineering and minimize harm for our users, while still building feature rich applications.

    Read More

    Posted 2020-04-04 11:40:00 CST by henriquez. 7 comments
  2. Obsessive Network LLC Launches Memespeech, An Unbannable End-to-End Encryption Format

    MINNEAPOLIS, MARCH 12, 2020 – FOR IMMEDIATE RELEASE

    ObSesSIVE nEtwoRk Llc HAS lAUNcHeD MemEsPeEcH, a NEw eNd-To-end EncryptiON FOrMAT thaT Is legaLly iMPoSSiBLE For the U.s. GOvernmeNT to bAN. MemESPEEch ENablES CONsUMErS tO hIdE EncryPTed MeSSAGES WITHin OrDInAry PaSSages oF FReE SPEECh. WItHoUT ThE DecrYpTIOn PASsWORd, iT iS IMpoSSiblE TO KNOw WhETher a PaSSagE OF mEmEspeEcH cOnTAinS AN ENCrYPTeD meSSAge, TherefORe thE U.s. GovernMENT cOULD Not bAN MEMESpEech WitHoUT BaNnInG SpEEch, ChEcKmate.

    IN TOdAY'S cOMPETiTIVE LaNDSCApe, thE U.S. cOngrESS hAS propOsEd A piecE OF leGISLAtioN CALLed tHe EarN iT Act, wHicH WouLD GIVe tHE U.s. ATTorneY GenerAL uNIlaTeRaL ConTRol OVeR HOw tecH COmPaniEs ImpLEMenT encRyptiOn TEchnOloGy. PURportEDlY AiMed AT StOppING peDoS FroM sHarINg kidDIe pOrn on facEbOOK (WhIcH has bECOmE a dIsGUSTing cESspOol OF FILth), tHE eARn iT acT HeRoicallY ENds aLL ChILD PoRN by bannINg All intERnet SecURItY.

    MeMEsPEeCh EnABles useRs whO sTiLL waNt secURiTy tO aDD THEir OWn ENCrYPtIOn BaCK iN to coMmUnICatIOn ApPs THaT aLLOw uSErs to ExcHAnGE teXT MessagES, thEreBy lEveraginG rEsoURcEs ANd addinG vAluE. as ThE maRkEt eVOlVEs, ComPAnIEs mAy bE foRCED to put BacKdooRS in thEIR own eNcryPtION, bUT they will nOT bE ABLE To IntErCEpt ThE ENCryPted MESSaGES CONsUmeRs SENd Over their netwoRkS witH mEmESpeEch. ThiS WILL CreAte nEw oPporTuNItiEs IN THe law eNfoRcemENT SECtoRs tO whine AbOUt nOt BeIng AblE to SPY On EVErythinG.

    MEMeSpEecH waS dEvElopeD by ObseSSIvE NEtWoRk lLc, A bOOTsTRaPPED UnICorn StARtUp WIth a $1.3 BIllIoN VaLuatIon aND A LeaDiNg-EdGe tEAM Of rocKsTAr nINJa 10X DEVeLoPErS wIth A CoLLECtiVE exPErience oF 75,000,000 years Of RUsT ANd goLang.

    "aftER COLlAboRaTINg WITh conSUmer rElATiOns AND brand EXperT teAmS AcRoss tHe INdustry, We ReaLIzeD THAt meMESpEECh CoULD rEVoLuTionizE ThE CONSUMER-suCCesS StRATegIes of OUr StAKeHOLDERs BY UsiNG BIG DATa machIne leaRNINg aI," saId heNRiQuez, sOME GuY At OBSesSiVe netWork.

    MeMeSpeECH is iMMeDIATeLy AvAilAbLE as a brOWSER eXTeNSiON foR MOzILla FIrefox anD GoOgle chROmE, As wEll as A dEvElOpER sPECIfiCatIoN, enAbLInG enGinEeRs tO DEVElOP mEmeSpEeCh implEMeNTATIon iN tHE LanGuAgeS oF THeir ChOicE.

    Posted 2020-03-12 16:20:00 PST by henriquez. 1 comment
  3. Cryptoops — a totally pointless JS crypto library

    I've recently been working on a project that involves encryption in web browser extensions, like take some text and encrypt it using a password. Pretty standard, except how do you do that in JavaScript? Due to my obsessive aversion to third party client dependencies, the decision was clear: I must read the specifications to all the relevant cryptographic standards and implement the algorithms myself in Javascript, from scratch.

    It never dawned on me that in the half-decade since I last played with JS crypto, perhaps things might have advanced past the previous status-quo of "roll your own crypto library #yolo". I neglected to consider the possibility that all of this functionality might be already baked into modern web browsers. Well turns out it is. While I was putting the finishing touches on my library (using a Web Worker to make it multi-threaded), I stumbled onto documentation for the Crypto.subtle API, a.k.a. everything I had just built but way faster due to being compiled code that takes advantage of native cryptographic CPU instructions. Oops.

    I can't bring myself to straight-up delete this code, but in reality it should never be used by anyone for any reason. Rather than try to make some sort of point (this is pointless), I'll leave it here for those who click "Read More."

    Read More

    Posted 2019-11-02 19:24:00 CST by henriquez. Comments
  4. The FBI’s plan to criminalize strong encryption

    Now that Congress has quietly backed away from CISPA and expansion of the CFAA, the Federal Government has wasted no time in introducing new half-baked Internet regulations. The latest comes courtesy of the FBI. Under their proposed expansion of CALEA, a federal wiretapping law, online service providers would be required to build wiretapping capabilities into their software, allowing law enforcement to secretly monitor user communications.

    According to the FBI, these expanded monitoring capabilities are required because child pornographers and terrorists are increasingly “going dark;” meaning that instead of calling each other on their wiretapped iPhones, they’re sending encrypted messages over the Internet, and the FBI can’t read them. By expanding wiretapping requirements to include online service providers, the FBI reasons, these tech-savvy villains can be brought to justice.

    On the surface, this looks like a reasonable proposition. But, as is the case with all technical regulations, the devil is in the details.

    For a typical online service provider, like Gmail, complying with a wiretapping order would be little to no trouble. Because a user’s messages are stored centrally on Google’s servers, Google could simply give the FBI access to their Gmail servers and be done with it.

    But some online service providers allow for the exchange of encrypted messages between users. Although the service providers may run central servers that facilitate the exchange of these messages, they are not readable by the service providers due to their encryption. Only the intended recipients can decrypt and read them. For these service providers, the only way to comply with a government wiretapping mandate is to bundle secret monitoring capabilities, or “backdoors,” into the actual apps that run on users’ computers or smartphones.

    While this might sound like a crazy conspiracy theory, it is the primary concern of a leading group of computer security researchers, including cryptography legends Bruce Schneier and Phil Zimmerman. Last Friday, the Center for Democracy and Technology released a report condemning the FBI’s plan. They warn that requiring software providers to install backdoors on peoples’ devices would “lower the already low barriers to successful cybersecurity attacks,” by giving hackers an easy way to attack apps while remaining undetected.

    But this could be exactly what the FBI wants. The FBI’s plan effectively gives developers a choice:

    • Install sketchy, easily-hacked monitoring software on your users’ devices.

    • Pay $25,000 a day in non-compliance fines. (unpaid fines double daily after 90 days, quickly paying off the national debt)

    • —or— just don’t bother offering end-to-end encrypted software.

    By making the first two options morally reprehensible and unrealistically burdensome, the FBI might hope that companies will just stop offering encrypted software for their users, making it much easier for them to centrally wiretap peoples’ communication.

    Certainly, investors will be less willing to fund startups that are required to install backdoors on users’ devices. If such a backdoor is exploited by hackers, where does the liability fall? It is arguably negligent to include a feature that is unquestionably adverse to every user, regardless of whether the service provider was required by law to do so. Unless the FBI also gives service providers immunity for any damages related to the backdoors, it’s quite likely these companies will be on the losing end of lawsuits when they inevitably get hacked.

    More importantly than concerns about stifling Silicon Valley innovation, the FBI’s proposed regulation raises questions about the rights of government to eavesdrop on its citizens. Since the 1990s, law-abiding people have taken for granted their ability to exchange encrypted digital communication with complete (or at least pretty good) privacy. Are child pornographers and terrorists a big enough threat to justify taking this away?

    I hate to end with a physical analogy, but this is a great way to explain the issue to someone less tech-savvy. Imagine you are a manufacturer of the locks used in bank and casino vaults. You take great pride in your craft, and your lock is secure against all but the most extreme attempts to break it. Now, suppose one day the FBI comes and tells you, “it’s fine and all that you built this vault, but we need you to install a second keyhole so we can open the vault and see if there are terrorists hiding inside.” The key they want to use is no more sophisticated than a house key, and can be opened by the most pedestrian of criminals. Is this really a good idea?

    Posted 2013-05-20 00:00:00 PST by henriquez. Comments
  5. Parsing OpenPGP Key Export Format with JavaScript

    If you tuned in last time to my post about scalar numbers and multiprecision integers, you might think that I'm just writing a book report for the OpenPGP spec. That’s not actually the case. My goal is to write a JavaScript app that allows encryption and decryption of OpenPGP messages, and I’m trying to document my progress as it happens. Reading the spec and obsessing over every definition will only get us so far. I’m tired of reading, so now it’s time to write some code.

    In this post, I will be walking through the process of reading OpenPGP public key files with JavaScript. Before we can get on to the more glamorous work of encrypting and decrypting OpenPGP messages, we need a way to get public and private keys into our app. And even before we can do this, we need a way of getting JavaScript to simply read data from the files themselves.

    OpenPGP data files and JavaScript

    OpenPGP data files—which can include public key, private key or encrypted messages (among other things)—can be stored as raw binary or as, what the spec calls, ASCII Armored data. “ASCII Armor” (see section 6.2 in the spec) is a fancy way of saying that the binary data is Base64-encoded and saved as ASCII along with a checksum and some particular headers. When you decode an ASCII Armored data file, you should get the exact same binary data as if you had opened an equivalent binary file. (That’s part of what we’re going to prove today.)

    Unfortunately, Javascript makes it a bitch to work with binary files, so that’s why I’m devoting a post solely to reading files into JavaScript and Base64 decoding ASCII Armor into binary. We’ll actually parse the binary into meaningful data structures at a later time. In order to properly decode ASCII Armored data, we’ll need JavaScript functions to Base64 decode/encode as well as a CRC24 checksum function. The spec helpfully describes how all of this will work, and we’re going to implement it!

    Export your Public Key files with GNUPG

    GnuPG is what all the cool kids use for PGP encryption these days, and ensuring interoperability with their export files and messages is a big priority for my project (although they can keep their license). If you don’t already have it set up, check out this excellent intro to OpenPGP by Zachary Voase, which includes installation instructions and usage tips for GnuPG. Once you have it set up and you have at least one entry in your public keyring (probably your own personal public key/subkey), export your keyring to both binary and ASCII Armored files using the following commands:

    gpg2 --export > pubkeys.gpg
    gpg2 --armor --export > pubkeys.asc

    Opening ASCII Armored keyfiles with JavaScript’s File API

    The previous commands exported two files: pubkeys.gpg, which is binary, and pubkeys.asc, which is ASCII Armored. Using a relatively new “HTML5” feature in JavaScript, (namely the File API) we can read the data from these files directly in the browser without first posting them to a server or anything ugly like that. This might not work on older browsers, so everyone using those browsers can cry me a river.

    In order to open a file with JavaScript, we need one of those file uploader form input elements. I shall give it the id ascii_keyfile.

    <input type="file" id="ascii_keyfile"/>

    Next we use JavaScript to bind an event to the file input so we can call a function when it changes. Note that I’m using MooTools for the event binding and document.getElementById shortcutting, but you can easily substitute your own favorite JavaScript framework as needed.

    window.addEvent('domready', function() {
        $('ascii_keyfile').addEvent('change', open_ascii_keyfile);
    });
    
    var open_ascii_keyfile = function() {
        var file = $('ascii_keyfile').files[0];
        read_file(file, parse_ascii_keyfile);
    }
    
    var read_file = function(file, callback) {
        var reader = new FileReader();
        reader.onload = function(evt) {
            var binary = evt.target.result;
            callback(binary);
        };
        reader.readAsBinaryString(file);
    }
    
    var parse_ascii_keyfile = function(data) {
        // ... ACTUALLY DECODE THE FILE HERE ...
    }

    This code binds a change event to the ascii_keyfile input. When you open your file via the uploader button, this event fires and triggers the open_ascii_keyfile function. This pulls the File object for the attachment out of the input and sends it into the read_file function along with the parse_ascii_keyfile callback function. Since reading data from the file may take awhile, it happens asynchronously. The read_file function starts this process and, when it completes, it sends the file’s data into the parse_ascii_keyfile callback.

    If all goes well, the file will be read and passed into parse_ascii_keyfile as ASCII text. It should look something like this:

    -----BEGIN PGP PUBLIC KEY BLOCK-----
    Version: GnuPG v2.0.19 (Darwin)
    
    mQENBFEN6EoBCADChZ+c6Q84tJ+WLTKYfhdN49OTUlxmoZD8cou6Bdi/EKXvpciA
    ydnD+SmlYf4pjAOwEiEsKJ6swLORAam4q0pnW9gAALbclhwDf9J4sLwUkh4F4D9P
    6TJX2vPEk4WRkudkj2TW3H2Wn1d7fQ3zlwLtK/bC5YeajuAIAk1m5zCtMbeZoYGc
    FWU+Max2G4Xr1/5JmUzfVtVSlxdJj7SX1FtJ/zj/eWklKNtl05yBWA+NyFpkgkzR
    DP+oJYBPdNoyS5mqNNIEnIIjDAUiufhGzkk2+865gIOH9X2WWCB5p0EGsR8ZzZA6
    H379WPca+GTlu5JncEi7lLcg+eQRwxQu9S6XABEBAAG0QEplZmYgTCAoaHR0cDov
    L3J1YmJpbmdhbGNvaG9saWMuY29tKSA8amVmZkBydWJiaW5nYWxjb2hvbGljLmNv
    bT6JATgEEwECACIFAlEN6EoCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJ
    EJmhufkmy7JniVYH/3Mjgo2gDDsc8tTPaIsBbYacB40pMOMX7+KxSQktrUZkGqwJ
    TlGnfBB4R8jz+32dBjX/OmeGYFTl9xFMBx+MuQlHW9Sl0ffV+Gpk9YbebBZaPn7Y
    5OpinF9e7zuFH7MyI72SIM7S1CvvfP3QrYj7viBitddJ+eW3Vx3ANgpkr8Bj9auH
    oT043dlfm/xpqozOLwbVM0BADJge0zQvNKGpoZjoHU2mNSSGhWhXAPCRp9wVOCCE
    SxbL+2Wi++ZUQUXO9DIxAQJy6HJfPx+PvBeedGAisfovNwtT0tDfJvyyPnRTKtEz
    TWiWDYwNUY80A6o/KkmSxSs5OeSh4t08phBIzWu5AQ0EUQ3oSgEIAMZf+w8pVqj/
    ZUQtacxzDe52kz+HtljJq4ltxulxQtoln5VkP5vWGq3uF1RFBoLVZ0OE/61yZixG
    8pOPMiGzHWJtidtQk7GxT/Z/b34voeTeruZjpfm3ty14sQvmApaRpjEQaNFTPy7d
    DiJKqGkD7teb/Mx8rtWJpN60hTiww1cOP5VjBvC82mn6uZ9DU2vJ6VwBTmwYnZMa
    XLiGRIpEAOqtLag4XwYrHS04H7No3asxSGhlyVN2KnxvlIMwoTZ+bTVaOr2ivCIC
    el1dY2kC5LsfMa04z2Ne7fme+pnGM62ufC+l/T9H58vsw1VFl5vanYmJugtFzxHF
    HzU3atdbHzEAEQEAAYkBHwQYAQIACQUCUQ3oSgIbDAAKCRCZobn5JsuyZ138CACu
    mdutchMDVE7V8ewhzsOCHgSMQjnmkB0HFCll2RxbhLz6x8SmzcQK107XbHQwFCdF
    A5v4JgFtwb6b9W9WShemNvC7tNx/loo2C+EiUKA9tURo/rJORu6S1jR79BaaOUUj
    MsB/jxxF2eRzE86SzgWXj34pYyoqJeMaiLSdXcCNW8eyN1i3gf8XpMlM7Ldv0Bq7
    vqbU2sDXBQvPDbNyhVIZjqfjTOBJl54NWHYRXlybFaSrXb7Qg/9ac+54TPpgCBTs
    1kR/HSZDujWE891NqlKGpSN4MDyi3WRL2RVbW0s5+8f8odNJuswIo1tWiNXBHVXs
    2/eCtlrSbyoTGYj0ErY0
    =a7UR
    -----END PGP PUBLIC KEY BLOCK-----

    There are two parts to this data that we care about. First we have the big block of ASCII characters grouped 76 to a line. That’s the actual data encoded as Base64. Then, the second to last line is 4 more Base64 characters preceded by an equal sign—in this case it’s =a7UR. That’s our CRC24 checksum (stay tuned for more on that).

    Isolating and decoding our Base64 data

    The first thing to do is to isolate the Base64-encoded payload data from the block above. We can do that with regular expressions:

    var parse_ascii_keyfile = function(data) {
        // Our data begins at the first character index preceded by a blank line.
        var body_begin_index    = data.search(/^(\r\n|\n|\r)/m) + 1;
    
        // Our data ends right before the checksum line which starts with an "="
        var body_end_index      = data.search(/^\=/m);
    
        // Both of these indexes need to exist for the file to be readable.
        if (body_begin_index == -1 || body_end_index == -1) {
            alert('This is not a valid ASCII-Armored OpenPGP export file.');
            return false;
        }
    
        // Pull the body out of the data and strip all newlines from it
        var body        = data.substring(body_begin_index, body_end_index);
        var body        = body.replace(/(\r\n|\n|\r)/gm, '');
    
        // Grab the checksum while we're at it...
        var body_checksum   = data.substr(body_end_index + 1, 4);
        ...

    Now we’ve isolated our Base64-encoded data into the body variable. It’s time to decode it! Some browsers natively support Base64 encoding and decoding respectively via the btoa(data) and atob(text) functions, but I don’t trust them. I’ve seen Base64 implementations that try to “helpfully” UTF8-encode/decode the incoming/outgoing data, and this will actually break our binary data. Plus I was geeking out on this awesome guide to Base64 in JavaScript (if you don’t know what Base64 is or roughly how it works, check it out), so I decided to write my own:

    var base_64 = {
        chars: 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/',
    
        encode: function(data) {
            var output = '';
            for (i=0, c=data.length; i<c; i += 3)
            {
                var char1 = data.charCodeAt(i) >> 2;
                var char2 = ((data.charCodeAt(i) & 3) << 4) | data.charCodeAt(i+1) >> 4;
                var char3 = ((data.charCodeAt(i+1) & 15) << 2) | data.charCodeAt(i+2) >> 6;
                var char4 = data.charCodeAt(i+2) & 63;
    
                output  +=  this.chars.charAt(char1)
                            +   this.chars.charAt(char2)
                            +   this.chars.charAt(char3)
                            +   this.chars.charAt(char4);
            }
            if (c % 3 == 1)
                output = output.substr(0, output.length - 2) + '==';
            else if (c % 3 == 2)
                output = output.substr(0, output.length - 1) + '=';
    
            return output;
        },
    
        decode: function(str) {
            var data = '';
    
            for (i=0, c=str.length; i<c; i += 4)
            {
                var char1 = this.chars.indexOf(str.charAt(i));
                var char2 = this.chars.indexOf(str.charAt(i+1));
                var char3 = this.chars.indexOf(str.charAt(i+2));
                var char4 = this.chars.indexOf(str.charAt(i+3));
    
                data += String.fromCharCode(char1 << 2 | char2 >> 4);
                if (char3 != -1)
                    data += String.fromCharCode((char2 & 15) << 4 | char3 >> 2)
                if (char4 != -1)
                    data += String.fromCharCode((char3 & 3) << 6 | char4);
            }
            return data;
        }
    }

    Now, getting back to our parse_ascii_keyfile function, Base64-decoding the data is as simple as:

    var decoded_body = base_64.decode(body);

    Computing the CRC24 checksum

    The previous code should return a string of 8-bit binary characters. But how do we know this is the correct string of 8-bit binary characters? That’s where our CRC24 checksum function comes in to play. The CRC24 checksum is sort of like a cheap hash function. Given a certain input (in our case the binary data), it should provide an output which is somewhat uniquely mapped to the input and consistent (non-random). This output is in the form of 24 binary bits. If we then Base64 encode the output, we should get 4 characters which precisely match the CRC24 checksum near the bottom of our ASCII Armored keyfile. If you’re confused, check out Section 6 of the OpenPGP spec—it explains this pretty well.

    The spec also includes a very helpful example of a CRC24 function written in C, which translates very easily to JavaScript:

    var crc24 = function(data) {
        var crc = 0xb704ce;
        var len = data.length;
        while (len--) {
            crc ^= (data.charCodeAt((data.length-1) - len)) << 16;
            for (i=0; i<8; i++) {
                crc <<= 1;
                if (crc & 0x1000000)
                    crc ^= 0x1864cfb;
            }
        }
        return number_to_binstring(crc, 24);
    }
    
    var number_to_binstring = function(bin, bits) {
        bits || (bits = 32);
        var text = Array();
        var i = (bits < 32 && bits > 0 && bits % 8 == 0) ? (bits / 8) : 4;
        while (i--) {
            if (((bin>>(i*8))&255) || text.length) {
                text.push(String.fromCharCode(((bin>>(i*8))&255)))
            }
        }
        return text.join('')
    }

    The major difference between this and the spec is my addition of the number_to_binstring function. In the crc24 function, JavaScript is performing a bunch of bitwise operations on the crc integer variable as it iterates over the 8-bit ASCII character codes associated with each byte of the input data. These operations are performed numerically, even though the data itself is in the form of a string (JavaScript is kind of janky here with its spotty support for ByteArrays). The number_to_binstring function simply converts the resulting 24-bit crc number value back to a string composed of 3 8-bit ASCII bytes.

    We previously read the ASCII Armor checksum into the body_checksum variable. Now we can take our decoded data and compute its checksum, then Base64-encode that checksum and compare to body_checksum. If it’s a match then we can do a victory dance, because the data was not corrupted and it decoded properly!

    var decoded_checksum    = base_64.encode(crc24(decoded_body));
    
    if (body_checksum != decoded_checksum) {
        alert('Checksum mismatch! (Expected '+body_checksum+', got '+decoded_checksum+')');
        return false;
    }
    // Our data decoded successfully
    ...

    Opening binary files is easier. The checksums should match!

    We can repeat the same basic steps as far as the read_file function to read a binary file into JavaScript. There’s no special decoding required once you get the file data, but one useful result is to compute the checksum. If you’re working with the binary version of the same key export as you opened previously in ASCII Armored format, then the checksum on the data should be the same. This is useful for proving this whole process is sane.

    After all this work, all we got was some lousy binary data in JavaScript. Actually reading the data and doing something useful with it is a whole ‘nother can of worms. But not to despair—we have to crawl before we can walk, and walk before we can run, and I really need some whiskey so I’m done for now.

    Posted 2013-02-03 00:00:00 PST by henriquez. Comments