Difference between revisions of "Development reference"

From Bitpost wiki
Jump to navigation Jump to search
Line 492: Line 492:
     # download latest boost, eg: boost_1_59_0  
     # download latest boost, eg: boost_1_59_0  
     m@wallee:~/development$ 7z x boost_1_59_0.7z
     m@wallee:~/development$ 7z x boost_1_59_0.7z
     cd boost_1_##_0
    rm boost && ln -s boost_1_59 boost
     cd boost
     # then patch .bashrc as instructed
     # IMPORTANT: then patch .bashrc as instructed
     # eclipse and server/nix/bootstrap.sh are customized to match
     #     cmake-###/clean.sh and cmake-###/build.sh are customized to match
To upgrade a project:
    #    (and older eclipse and server/nix/bootstrap.sh)
To upgrade a CMake project:
    cd cmake-###/
To upgrade an older autotools project:
     cd nix
     cd nix

Revision as of 00:24, 12 September 2017

Design, programming and version control.

Major Objects
Continuous Integration
Architecture of major open source apps
Model View Controller
String escape formatting across different languages and systems
Web design mantra
    1. respect the input device
    2. detect it and use a reasonable default guess
    3. allow manual override
    1. high-res = SHOW LOTS OF DETAIL
    2. responsive, zoomable
git central shared repository


To configure the repo as shared:

git config core.sharedRepository true


git init --shared=group
Occam's razor
Deleting multiple elements in a collection

See MemoryModel::saveDirtyObjectsAsNeeded() for example code.

c++ Create a portable C/C++ project
Use QtCreator + CMake, both available everywhere!
   set up:
           pick the PARENT and use the name of the existing folder
           it creates CMakeLists.txt (THIS IS BASICALLY THE PROJECT FILE!)
           also creates main.c or something
           build it!  make sure it works
               TARGET_LINK_LIBRARIES(${PROJECT_NAME} pthread websockets)
           make sure it builds
           now we want a DEBUG build as well!
               Rename: release
               Add: Clone: release, name: debug
                   for debug, add a custom build step BEFORE make step:
                       command: /usr/bin/cmake
                       args: -DCMAKE_BUILD_TYPE=Debug .
                       working dir: %{buildDir}
                       (add it, and move it up above make)
                       (build em both)
               clone it into "{...} debug"
               change working dir to match Build path
               change any needed params
c++ in-memory storage of "major" objects

   Consider An Important Qt Design: QObjects cannot normally be copied
       their copy constructors and assignment operators are private
       why?  A Qt Object...
   isn't this true of all serious objects?  pretty much



   unordered_set works even better when combining frequent CRUD with frequent lookups
   Having multiple indices into those objects is also essential.
   Using unordered_set with object pointers and is very powerful.
c++ stl reverse iterator skeleton
From SGI...
reverse_iterator rfirst(V.end());
reverse_iterator rlast(V.begin());

while (rfirst != rlast) 
    cout << *rfirst << endl;
c++ stl reading a binary file into a string
   std::ifstream in("my.zip",std::ios::binary);
   if (!in)
      return 0;
   unsigned long length = in.tellg();
   string str(length,0);
       std::istreambuf_iterator< char >(in) ,
       std::istreambuf_iterator< char >() ,

For more, see c++ stl reading a binary file

C/C++ best-in-class tool selection
  • Compiler selection
    • linux and mac: gcc
    • windows: Visual Studio
  • IDE selection
    • linux and mac: Qt Creator
  • Debugger selection
    • linux and mac: Qt Creator
 class HttpsServer : public Server<HTTPS>
       // Call base class
   using Server<HTTPS>::Server;
c++11 containers



(the default will be by pointer address).

   map vs key redux
           + small key
           + encapsulation
           - requires mutable to solve the const problem
       use a key in the set, key includes a mutable object
           + encapsulation
       use a map
           + small key
           + solves const problem bc value is totally mutable by design
           + simpler equal and hash functions
           create an object with internal key(s)
           create an object with internal key(s)
               that proves difficult when dealing with member vars that are references
               nope, can't reassign refs
               the solution: use pointers not references
               yes that's right
               just do it
               two reasons to use pointers:
                   dynamically allocated
                   reassignment required
               there ya go.  simple.  get it done. 
Best solution is an unordered set of pointers:
typedef boost::unordered_set<MajorObject*> MajorObjects;
            m1.id_0 == m2.id_0
        &&  m1.id_1 == m2.id_1;
static std::size_t hash_value(MyKeyObject const& mko)
    std::size_t seed = 0;
    boost::hash_combine(seed, mko.id_0);
    boost::hash_combine(seed, mko.id_1);
    return seed;


c++11 example for set with specific sorting
struct customers_set_sort_functor
        // the id is the key
        return l->id_ < r->id_;
typedef set<MyObject*,myobject_sort_by_id_functor> MyObjectsById;
c++11 loop through vector to erase some items


    if (*it.no_good()) 


for (auto& mc : my_container)
for (const auto& cmc : my_container)
    cout << cmc.asString();


#include <boost/range/adaptor/indexed.hpp>
    cout << element.value() << element.index();
c++11 for loop using lambda
vector<int> v;
   cout << val;
} );


c++11 integer types




FAST types:

   int_fast16_t                fastest signed integer type with width of
   int_fast32_t                at least 8, 16, 32 and 64 bits respectively
   uint_fast16_t               fastest unsigned integer type with width of
   uint_fast32_t               at least 8, 16, 32 and 64 bits respectively

SMALL types:

   int_least16_t               smallest signed integer type with width of
   int_least32_t               at least 8, 16, 32 and 64 bits respectively
   uint_least16_t		smallest unsigned integer type with width of
   uint_least32_t		at least 8, 16, 32 and 64 bits respectively

EXACT types:

   int8_t                      signed integer type with width of
   int16_t                     exactly 8, 16, 32 and 64 bits respectively
   int64_t                     (provided only if the implementation directly supports the type)
   uint8_t                     unsigned integer type with width of
   uint16_t                    exactly 8, 16, 32 and 64 bits respectively
   uint32_t                    (provided only if the implementation directly supports the type)


   intptr_t                    integer type capable of holding a pointer
   uintptr_t                   unsigned integer type capable of holding a pointer 
C++11 scoped enumeration
enum class STRING_PREF_INDEX int_fast32_t: { ... };



Old skool is still cool:

typedef enum
    // assert( SP_COUNT == 2 );
    SP_FIRST = 0                ,
    SP_ANOTHA                   ,

boost release and debug build for linux
   # download latest boost, eg: boost_1_59_0 
   m@wallee:~/development$ 7z x boost_1_59_0.7z
   rm boost && ln -s boost_1_59 boost
   cd boost
   # IMPORTANT: then patch .bashrc as instructed
   #     (and older eclipse and server/nix/bootstrap.sh) 

To upgrade a CMake project:

   cd cmake-###/

To upgrade an older autotools project:

   cd nix
   make clean      # removes .o files etc
   cd ..
   ./bootstrap force release
   ./bootstrap force debug
boost release and debug build for Windows
Open a VS2015 x64 Native Tools Command Prompt.


cd "....\boost_1_59_0"
(old stuff)


   virtual bool url_upgrade_any_old_semver(string& url)
     // Only then do we take action.
     boost::smatch sm_res;
       if (incoming_semver != semanticVersion())
         return true;
     return false;


C++ libraries


C++ https libraries

Configure Qt development on Windows + Mac + linux

Build the TagLib library with Visual Studio 2013

C/C++ building/linking
gcc makefile pain

It gives warnings like this:


and outright errors like this:

mv -f $depbase.Tpo $depbase.Po




C/C++ debugging
Qt Creator conditional breakpoint
To break on line number with string condition:
  • In the breakpoints list, rclick Edit
Qt Creator and linux debug libraries
 Install the project...
 -- Install configuration: "DEBUG"
 -- Installing: /usr/local/lib/libwebsockets.a
       Qt Creator->Options->Debugger->
           [ ] Show a message box when receiving a signal
           Debugger Helper Customization: handle SIGPIPE pass nostop noprint
QT Creator valgrind EASY
c/c++ gdb debugging
(gdb) help break
Set breakpoint at specified line or function.
If an address is specified, break at that exact address.
This is useful for breaking on return to a stack frame.


Library handling in linux

Tools to check library dependencies:

  • ldd [executable]
  • objdump -x [object.so]
  • readelf -d [exe]
  • sudo pldd <PID>
  • sudo pmap <PID>

Cross Compiling

C - Create a portable command line C project in Visual Studio
   Visual Studio: File -> New -> project
   Visual C++ -> Win32 -> Win32 Console Application
   name: oms_with_emap
   change the project to explicitly say "Not using precompiled header"
   remove the f'in stdafx.h
   recompile!  should be clean
   vs will recognize C files and compile accordingly
Single Page Application
    height: 100%; 
    overflow: auto;


  <-- header -->

  <div class="frame" id="frame1top">
    <div class="mdm-scrollable-div">

  <-- footer -->


  // We need to adjust layout on resize, in ready handler, and programmatically as needed.

  $( document ).ready(function() {

    // We need to adjust in ready handler, on resize, and programmatically as needed

    /* your other page load code here*/


  function adjustLayout(){

      // Fundamental dimensions
      var hh = $('#theheader').outerHeight();
      var fh = $('#thefooter').outerHeight();
      var workspace_height = $(window).height() - hh - fh;
      var workspace_width = $(window).width(); 
      var cols = 1;
      var col_gap = 16;
      // Margin is based on percent of one column width, so it goes to zero before column width does.  :-)
      // Width is based on total width minus margins, then split between columns.
      var margin = ($(window).width() / cols) * .04;
      var w = ($(window).width() - (margin * (cols + 1))) / cols;
      var h1 = workspace_height;
          left: margin * 1 + w * 0,
          top: hh,
          width: w,
          height: h1
Button handlers
Buttons can be represented with labels; in that case, use a click handler:
    jQuery('<label id="'+cycle.apsID+'" class="btn">MahButt</label>')
    .on('click', function(){

You can also get the checked state, and prevent the button from being pressed, among other things:

      jQuery('<label id="'+cycle.run+'" ...
      .on('click', function(e){
        var run = this.id;
        // Look for the (label->input)
        if (this.firstChild.checked)
        // To prevent checked/pressed state if desired:

Button bars are represented by labels wrapped around inputs:

    <div class="btn-group live-buttons" data-toggle="buttons"><div class="btn-group"><label class="btn"><input type="checkbox">text...

In that case use a change handler on the input:

      var label = jQuery('<label class="btn btn-sm btn-'+color+'"></label>').appendTo(action_button_bar);
      var input = jQuery('<input class="run-'+cmd+'" type="checkbox" autocomplete="off" value="'+cycle.run+'">').appendTo(label)
      .on('change', cycle.owned? function(){
        patchPick(this.value,'{ "action" : "run-hold"       }');
      } : function() {
        patchPick(this.value,'{ "action" : "run-buy"        }');
      var text = jQuery('<span class="glyphicon glyphicon-'+glyph+'"></span> <span class="hidden-xs">'+cmd+'</span>').appendTo(label);

If you just need clicks from the button bar, you do NOT NEED input:


Then you can put the change handler right on the label:

    var applybar = jQuery('<div id="applybar-'+cycle.run+'" class="btn-group pull-right" data-toggle="buttons" />');
    var apply_button = jQuery('<label id="apply-'+cycle.run+'" class="btn btn-sm btn-moneygreen"><span class="glyphicon glyphicon-ok"></span><span class="hidden-xs"> Apply</span></input></label>')
    .on('click', function(e) {    
        // Do work

Programmatically pressing a button
  • Set the active class on the button/label
  • Set the input to checked
<label class="btn active">
  <input type="checkbox" autocomplete="off" checked>
  Click me
Handling the UNCHECK event on a pushbutton
Again you MUST SET BOTH active and checked STATES PROPERLY when creating the button (see above). Do not set either if the button is unpressed; set both if it is pressed.

Then you can use a single change event:

  $("input[type='checkbox'][class='run-hold-on-buy'     ]").change(function() { patchPick(this.value,'{ "action" : "'+(this.checked?'hold-on-buy-on'     :'hold-on-buy-off'     )+'"}');  });
Collapsible panel
  • Use a unique id on content div, data-target to connect panel header to content, mdm-panel-collapser so javascript can find icon, abt-scrollable-panel for margin.
  • HTML for each panel (starts collapsed)
    <div class="panel panel-default abt-scrollable-panel">
      <div class="panel-heading collapsed" data-toggle="collapse" data-target="#my-content-block-one">Software development skills<span class="mdm-panel-collapser text-muted glyphicon glyphicon-chevron-down pull-right"></span></div>
      <div class="collapse" id="my-content-block-one">
        <div class="mdm-panel-body">
         <!-- CONTENT, can include another nested panel, just add .mdm-nested-panel to class; example: -->
          <div class="panel panel-default mdm-scrollable-panel mdm-nested-panel">
            <div class="panel-heading collapsed mdm-job-panel-heading" data-toggle="collapse" data-target="#toshiba-job"><p><strong>Senior Developer and Offshore Manager</strong><i> - Toshiba Global Commerce Solutions, Inc.</i><span class="mdm-panel-collapser text-muted glyphicon glyphicon-chevron-up pull-right"></span></p><p><small>April 2014 – August 2015</small></p></div>
            <div class="collapse in" id="toshiba-job">
              <div class="mdm-panel-body">
                <!-- SUBCONT(IN)ENT im so funny -->

  • For an expanded panel, simply change icon from down to up, and add "in" to content div:
    ... glyphicon-chevron-up ...
    <div class="collapse in mdm-panel-body" id="collapseOrderItems1">
  • Define Javascript collapser once on load
 $('.collapse').on('hide.bs.collapse show.bs.collapse', 
  • CSS
   .mdm-nested-panel {
     margin-top: 1em;
     margin-left: 1em;
   collapse.mdm-panel-body collapsing.mdm-panel-body {
       margin: 1em;
  • Common reusable function:
 function toggleCollapser(e) {
         .toggleClass('glyphicon-chevron-down glyphicon-chevron-up');
   // Prevent bubble up to any parent collapsers.
   // This allows nested collapsers, whoop.
ALWAYS initialize incoming data
Hard-earned data-import lessons:
  • tsv() will do complete date formatting, so when you remove it to use JSON directly, you HAVE TO convert JSON timestamps of ANY kind to full Javascript date objects:
 var data = [{"date":"2017-04-01T04:00:00.000Z",...
 // JSON VALUES ARE ALWAYS STRINGS, we need to change to Javascript DATE!
 // Incoming datasets need a bit of massaging.
 // We do that in a function so we can reuse it on incoming dataset updates.
 function initializeDataset(dataset) {
     // TIME ON X
     // Convert date strings to actual date values.
     // MDM MAKE SURE this matches the incoming format.
     // Adding date so we can run the chart across days without trouble.
     // var parseDate = d3.time.format("%H:%M:%S %m-%d-%Y").parse;
     // var parseDate = d3.timeParse("%Y %b %d");
     var parseDate = d3.utcParse("%Y-%m-%dT%H:%M:%S.%LZ");
     dataset.forEach(function(d) {
         // there is NO SKIPING THIS STEP, you have to get valid Javascript date objects out of JSON strings
         d.date = parseDate(d.date);
         // we WOULD have to divide all data by zero,
         // but we already grabbed post-data that was already converted
         // This WORKS but makes data / 10000 (very small numbers)
         //for (var i = 1, n = dataset.columns.length; i < n; ++i) d[dataset.columns[i]] = d[dataset.columns[i]] / 100;

  • tsv() will create an ARRAY but it also jams in a columns PROPERY; it takes two steps to duplicate that:
  var data = [
    {"date":"2015-06-15T04:00:00.000Z","Google Chrome":0.48090000000000005,...},
    {"date":"2015-06-22T04:00:00.000Z","Google Chrome":0.48979999999999996,...),
 data.columns = ["date","Google Chrome","Internet Explorer",...  ];
ALWAYS set range and domain properly
You have to know your display size (range) and the min and max of your data (domain), on each axis. From Mike's docs:

D3’s scales specify a mapping from data space (domain) to display space (range).

       D3’s scales can also be used to interpolate many other 
       types of display-space values, such as paths, color spaces 
       and geometric transforms.
         var x = d3.scale.linear()
             .range([0, 420]);
       Although x here looks like an object, it is also a function 
       that returns the scaled display value in the range 
       for a given data value in the domain. 
       For example, an input value of 4 returns 40, and an input value of 16 
       returns 160. To use the new scale, simply replace the 
       hard-coded multiplication by calling the scale function:

Watch out for Mike's examples where he (trickily) doesn't bother to set a domain because his data is in the [0..1] set, and that's the default domain, apparently.

RESTFul http
  • post - to push a form or JSON to the server
Javascript and JQuery
Formatting examples (numbers dates etc.)
 // Javascript sucks at dates.
 // d3 to the rescue!  
 var parseInitDate = d3.utcParse("%Y-%m-%dT%H:%M:%SZ");
 var initDate = parseInitDate(initial_managed_value_date);
 var initDateString = 
     +"/"+ (initDate.getUTCMonth()+1) 
     +"/"+ initDate.getUTCDate()
     + " " + initDate.getUTCHours() 
     + ":" + initDate.getUTCMinutes() 
     + ":" + initDate.getUTCSeconds();
 var rightNow = new Date;
 var initToNow_DecimalDays = parseFloat(rightNow - initDate) / 864e5;  // 86,400,000 ms/day
 var initToNow_PercentOfYear = initToNow_DecimalDays / 365.0;
 var change_in_value = (parseFloat(total_managed_value) - parseFloat(initial_managed_value))/parseFloat(initial_managed_value);
 $('#change_in_value').val((change_in_value*100.0).toFixed(1)+ "%");
 $('#equivalent_apr').val((change_in_value/initToNow_PercentOfYear*100.0).toFixed(1)+ "%");
Array vs Object lifetime
       // If the JSON data were an array, we would need to slice-to-clone to keep it around:
       //     var dataCopy = data.slice();
       //     updateStockData(run,dataCopy);
       // We have an object so we can pass a reference, and it will keep the object around.
Event handlers
Three basic methods:
  • Attach a function to an event of a specific element; charts.js example:
     } : function() {
     var text = jQuery(' ').appendTo(label);
  • Add a function handler once, for a class of elements; picks.js example:
 $("input[type='checkbox'][class='run-top'    ]").change(function() { patchPick(this.value,'{ "action" : "run-move-top"   }');  });
  • Event delegation: Javascript takes events and "bubbles them up" through the chain of parents; set up a handler on a parent to listen for these bubbles.
  • At end of handler, prevent further bubbling with stopPropagation() - still finishes the current event (eg following the href of an <a>):
    var datepicker = jQuery(
      '<div class="input-group date form-inline pull-left" data-provide="datepicker" id="datepicker'+cycle.run+'">'+
        '<input type="text" class="input-sm form-control">'+
        '<div class="input-group-addon">'+
            '<span class="glyphicon glyphicon-th"></span>'+
    .on('changeDate', function(e){
      alert('next '+cycle.run);
  • Or use preventDefault() to stop completely:
    var datepicker_next = jQuery(
      '<a href="">My javascript action</a>'
    .on('click', function(e){
      alert('next '+cycle.run);
Debug clicks on ANY PAGE
  • Press F12 => Sources => Event Listener Breakpoints righthand pane => Mouse => [x] click

B O O M we have it captured and can step into ANYTHING.

JWT flow
Client is easy!
   Client application                                            API
        |                                                      |
        |                   GET /api/employees                 |
        |                     403 Forbidden                    |
        |                                                      |
        |                                                      |
        |                 POST /api/authenticate               |
        |     { login: "john.doe", password: "password" }      |
        |                      200 Success                     |
        |             { token: "my.personal.token" }           |
        |                                                      |
        |                                                      |
        |                 GET /api/employees                   |
        | Header { "Authorization: Token "my.personal.token" } |
        |                      200 Success                     |
        |                                                      |
Node.js installation
Node create a new module
Register with npm
A one-time registration is required on a new machine if you want to publish from it:
npm adduser
Username: moodboom
Password: (see private)
Email: (this IS public) moodboom@gmail.com
Publish a node module
sudo npm install -g # keep doing this until you are happy with local install
# update version in package.json
# this creates a FULL "annotated" tag, not a "lightweight" tag that doesn't show up for [git describe] - it also removes the need for a separate commit
git tag -a 1.0.5 -m "changes include..."
git push && git push --tags  # NOTE: bitpost has a git hook to push changes all the way up to github
npm publish
Update a node module's dependencies
# make sure dependency in package.json has a carat at the beginning of its version (^x means "at least" version x)
# make sure the dependency has a new version available - completely publish it first if it is your own
# then you can simply reinstall from within the module folder to get all dependencies upgraded
sudo npm install -g
Develop several node modules at once
Convert dependencies to use local packages instead of published versions, eg:
cd ~/development/mah-haus
npm install -S /home/m/development/thedigitalage/rad-scripts

Then reinstall everything (local dependent modules, then parent modules, pita - consider links if doing longer-term dev)

sudo npm install -g

Then convert back to published versions as they become available (it's up to me to stabilize and publish new module versions):

cd ~/development/mah-haus
npm install -S rad-scripts
auto AWS
  • npm install -g aws-sdk
  • Add credentials here: C:\Users\Administrator\.aws
  • see existing scripts, anything is possible
install bootstrap
  • npm install -g grunt-cli
  • mkdir mysite && cd mysite
  • npm install bootstrap
  • cd node_modules/bootstrap
  • npm install # to actually pull down dependencies
  • grunt dist # builds and minifies so you're good to go!
 /* class="first second" */
 .first.second {}
 .first, .second {}
 .second {}
 .parent .child {}
 .parent > .child {}
Sqlite timestamp-to-readable-date query
php debugging
Tail these:
tail -f /var/log/apache2/sitelogs/thedigitalage.org/ssl_error_log
This leads to too much noise, not needed...
emacs /etc/php/apache2-php5.3/php.ini
  display_errors = On
/etc/init.d/apache restart
Bash basics but please prefer node or python :-)
SVN repo move across servers
  • use a tool like VisualSVN to stop SVN server
  • use a tool like VisualSVN to restart SVN server


SQL Server 2008+ proper upsert using MERGE
       -- There are several options to do that.
       -- Trying to do it correctly means...
       -- Cest la vie.
       USING (SELECT @ID AS ID) AS new_foo
             ON rpmr.ID = new_foo.ID

               @last_months_year as DCSRYear,
               @last_month as DCSRMonth,
               @last_month_name as MonthName,

                   SET f.UpdateSpid = @@SPID, 
                   UpdateTime = SYSDATETIME() 
Web Services
Firefox Addon development


  • main.cpp
  • MVC code
  • nix/copy_from folder
  • make sure .bashrc is configured for boost
  • nix$ bootstrap_autotools_project.sh force release debug
  • set up eclipse according to screenshots in Eclipse


int main( int argc, char * argv[] )
    return 0;

Then in Visual Studio...

File->New->Project from existing code
Project file location:  <base>\project_name
Project name: project_name
[x] Add files from these folders
   Add subs  
   [x]        <base>\project_name
Use Visual Studio
  Console application project


INCLUDE: C:\Software Development\boost_1_53_0