Loading...

Scalable
3D molecular graphics
on the web

MMTF & NGL Viewer

Alexander Rose, Anthony Bradley, Yana Valasatava,
Jose Duarte, Andreas Prlic, Peter Rose

3Dsig 2016

Atoms in the Protein Data Bank


In over 120.000 structures (asymmetric units)

Here is one of those atoms.

However they are usually not alone ...

... they come in groups, like this heme ...

... and form proteins ...

... as well as large complexes ...

... are those (becoming) too big for browsers?

Growth of the PDB archive

  • Largest structure: HIV-1 Capsid
    • ~2.4M unique atoms
    • gzipped mmCIF file: 48.7MB

  • 68 of the 100 largest structures
    deposited in past 3 years


How to display such structures in a web-browser?

Steps to display a structure


  1. Download File
  2. Decompress & Parse
  3. Populate Data Model
  4. Create Geometry
  5. Render

Download & Parsing

  1. Download File ➠ ???
  2. Decompress & Parse ➠ ???
  3. Populate Data Model
  4. Create Geometry
  5. Render

Speeding-up
Download & Parsing

  • Download: Smaller file-size
  • Parsing: Binary encoding
  • Consistent & complete data

➠ A new file format for transmission?

MacroMolecular
Transmission Format (MMTF)

  • Binary (MessagePack as a container)
  • Bespoke compression strategies:
    • Dictionary encoding → store once, reference repeatedly
    • Run-length encoding → store value + number of repeats
    • Integer encoding → store floats with reduced precision
    • Delta encoding → smaller numbers by storing differences
    • Integer packing → tightly pack small integers

File size: mmCIF vs MMTF

File size of the whole archive
~120.000 structures, gzip compressed

Parsing time: mmCIF vs MMTF

Parsing time of the whole archive
~120.000 structures, gzip compressed
on a 2.6 GHz Intel Core i5 using JavaScript

MMTF - Included Data Fields

  • Model/chain/group/atom names and ids
  • Coordinates & B-factors
  • Complete bonding data
  • DSSP secondary structure
  • and more, see http://mmtf.rcsb.org/

MMTF - Flavors

full (green), reduced (red)
  • Full:
    • All atoms
    • Three decimal place precision
  • Reduced:
    • C-alpha/phosphate & ligand atoms
    • One decimal place precision
  • Build your own:
    • Specification: required & optional fields
    • Planned: user data fields

MMTF - Availability

  • Open Source specification and libraries:
    Java, JavaScript, Python, (C/C++ planned)
  • Weekly updated PDB archive:
    full & reduced flavor


Website: http://mmtf.rcsb.org/
Poster: #21 (Anthony Bradley)

Download & Parsing

  1. Download File ➠ MMTF
  2. Decompress & Parse ➠ MMTF
  3. Populate Data Model
  4. Create Geometry
  5. Render

Efficient Storage & Access

  1. Download File ➠ MMTF
  2. Decompress & Parse ➠ MMTF
  3. Populate Data Model ➠ ???
  4. Create Geometry
  5. Render

Data Model Wish-list

  • Fast creation from parsed file
  • Memory efficient
  • Convenient access
    (e.g. of atom/residue level data)

Columnar Stores

Row-based:
   ASN, 1, C | GLY, 2, H | PRO, 3, H | VAL, 4, H
Column-based:
   ASN, GLY, PRO, VAL | 1, 2, 3, 4 | C, H, H, H


  • Single TypedArray per property
  • Parsed MMTF data can be copied in blocks
  • Proxy objects for row-like access

Columnar Stores

Efficient Storage & Access

  1. Download File ➠ MMTF
  2. Decompress & Parse ➠ MMTF
  3. Populate Data Model ➠ Columnar Stores
  4. Create Geometry
  5. Render

Geometry

  1. Download File ➠ MMTF
  2. Decompress & Parse ➠ MMTF
  3. Populate Data Model ➠ Columnar Stores
  4. Create Geometry ➠ ???
  5. Render

Impostors

Impostors: For each pixel let GPU test intersection of sphere and camera ray.
Quality resolution independent. Also available for cylinders.

Instancing


  • Create geometry once
  • Send to GPU once
  • Transform position &
    render multiple times


⇦ Green surface is reused 59 times
for highly symmetric virus capsid
(PDB ID 1RB8)

Geometry

  1. Download File ➠ MMTF
  2. Decompress & Parse ➠ MMTF
  3. Populate Data Model ➠ Columnar Stores
  4. Create Geometry ➠ Impostors & Instancing
  5. Render

3D Rendering

  1. Download File ➠ MMTF
  2. Decompress & Parse ➠ MMTF
  3. Populate Data Model ➠ Columnar Stores
  4. Create Geometry ➠ Impostors & Instancing
  5. Render ➠ ???

Java Applet Deprecation

  • Java Applets have provided
    fast execution and GPU access
  • Removed from Google Chrome
    in version 45 (Sep 2015)
  • Oracle to deprecate Java plugin
    in upcoming JDK 9

Browser Advances

  • JavaScript approaches native speed
  • WebGL offers plugin-free access
    to the graphics card

webglstats.com

3D Rendering

  1. Download File ➠ MMTF
  2. Decompress & Parse ➠ MMTF
  3. Populate Data Model ➠ Columnar Stores
  4. Create Geometry ➠ Impostors & Instancing
  5. Render ➠ WebGL
With all that in place

Does it scale?

We can test it right now as
this presentation is running in a browser

HIV-1 capsid
hexameric subunit, 10800 atoms

HIV-1 capsid
216 hexameric and 12 pentameric subunit, ~2.4M unique atoms

PBCV-1 virus capsid
1680 instances of 9693 atoms (~16M atoms)

Faustovirus major capsid
2760 instances of 14478 atoms (~40M atoms)

It does scale!

NGL Viewer




AS Rose & PW Hildebrand. NGL Viewer: a web application for molecular visualization.
Nucl. Acids Res. (1 July 2015) 43 (W1). doi:10.1093/nar/gkv402

Summary

  • MMTF
    • Small, compressed file-format
    • Fast to parse, libraries available
    • Useful data, like bonds, included
    • Enables routine analyses of the whole PDB archive

  • NGL Viewer
    • Memory-efficient data model, MMTF support
    • Fast rendering through WebGL
    • Embeddable library, API
    • Enables scalable 3D molecular graphics on the web

Acknowledgements


  • RCSB PDB Team
  • NCI/NIH
    (U01 CA198942)

Thank you for your attention!


Questions?