A follow up on my previous work with crime by police beat, adding the ability to switch between beat and district. A district is a much larger area and changes the perception of the underlying data set.
View the demo
A follow up on my previous work with crime by police beat, adding the ability to switch between beat and district. A district is a much larger area and changes the perception of the underlying data set.
View the demoEver since my first programming experience with AutoLISP, the variant of the LISP language intended for automating AutoCAD, I had wanted to do data analytics again. I have used Javascript to process data, with the main disadvantage being that it is a very resource intensive method and not exactly what Javascript was intended for when it was developed. I had heard about the R language dedicated towards statistical analysis and decided a distributed approach was warranted: using R to process my data sets and create data files, and a web front end for actually displaying the data.
Initially my goal was use a React based visualization library, but after encountering issues with the only one in widespread use dedicated towards geographic JSON, I decided to use D3.js with React simply managing my interface state.
View the demoI began to actively look at the new implementation of Javascript, based on the ES6 spec, around June of 2016 after attending a related meetup about NodeJS. I had heard about languages like CoffeeScript that could compile to browser-compatible Javascript as well as tools like Babel for compiling ES6 to Javascript, but I had never touched any code with the newest spec until I started to explore the latest Javascript tutorials. That's when I started to encounter unfamiliar syntax, even in tutorials with "vanilla" Javascript executed in the browser. At this point I became aware that the long awaited updates to the Javascript spec had made it into the browser engines.
The past few Javascript exercises sets I have done over the past year were exclusively pre-ES6 syntax, although sometimes I would end up using methods that had some support gaps, the string includes() method being one example (Internet Explorer did not embrace this method unlike most browsers). With all the advances in Javascript over the last few years, I felt that a newer approach was needed if I was going to brush up on the fundamentals while keeping up to date on new concepts.
My first exercise involved tying together keyboard input with audio output. One of the big advantages for string construction that I discovered here was variable interpolation. I found it much easier to place variables "inside" a string with delimiters compared to the old style of concatenating fragments and variables.
Old".my-section-class" + " ." + propertyName
`.my-section-class .${propertyName}`
'{} {}'.format('one', 'two')
Once I was introduced to LESS styling I used variables as a matter of course, but in doing so I left exploration of CSS behind. Only recently did I use css variables in an exercise. After having used jQuery style functions for so long it was illuminating to see exactly how style can be manipulated with vanilla javascript.
Compared to changing styles via class changes I found that manipulating a variable inherited by other properties was much simpler. The result was replacing a chunk of code into functionality that is "under the hood".
When it comes to processing data sets I have often used the Underscore library and it's functions like map() and filter(). As part of my vanilla JS refresher I went back to the core Javascript functions for handling data sets. This is one area where the fat arrow (=>) started to demonstrate its benefits to me regarding concise code. Instead of having to deal with functions that returned booleans, I could rely on the implicitness of the syntax (return if true).
BeforeAfterconst genX = castaways.filter(function(castaway) {if (castaway.year >= 1963 && castaway.year <= 1982){return true;}});
const genX = castaways.filter(castaway => castaway.year >= 1963 && castaway.year <= 1982);
In a way this makes code less human readable by putting more code into symbols. Also, it does look a little confusing to have an function call that looks like a comparison modifier. I will of course use it whenever appropriate, but I can understand people being puzzled by this new feature.
Reviewing the sort() function I decided to look into exactly how the comparison is done. I have read the basics on sorting algorithms in the past, so I decided to log the values being analyzed. Reviewing the purpose of the returns was a good refresher (1 to increase index, -1 to decrease index) and the end result was that sorting felt less like a black box.
The reduce() array function was one I hadn't used in previous projects. At first glance it seems like a simple way to aggregate the data in an array and return a sum. This could be done with a a foreach() method, but as I read the documentation further I came to understand why the different array methods exist. Stepping back and taking a moment to look at the specifics of each method, the advantage of reduce (it returns an object or value) made it better suited for aggregation than a foreach() method since the accumulation is handled by the method itself. This contrasts with having separate variable and updating that during the iterations of a foreach().
After completing a React build using Webpack, I wanted to get back to the build system I had first used when I began web development, Gulp. My previous experience with Gulp was limited to minification and LESS compilation, two relatively straightforward processes. I quickly realized that creating a React build would require much more development as I needed to package a framework with all of the necessary dependencies
var bundler = browserify({entries: ['app/index.jsx'], debug: true}).transform(babelify);
gulp.task('watchify', function () {var args = merge(watchify.args, { debug: true })var bundler = watchify(browserify('app/index.jsx', args)).transform(babelify)bundle_js(bundler)bundler.on('update', function () {bundle_js(bundler)})})
Aftergulp.task('compile-less', function () {gulp.src('app/styles/app.less').pipe(less({sourceMap: {sourceMapRootpath: '' // Optional absolute or relative path to your LESS files}})).pipe(gulp.dest("dist/styles"));});
gulp.task('compile-less', function () {gulp.src('app/styles/app.less').pipe(sourcemaps.init()).pipe(less()).pipe(sourcemaps.write(".")).pipe(gulp.dest("dist/styles"));});
The goal with this exercise was to learn the basics on how to create a 3D scene with geometry. I became familiar with the concept of meshes being composed of a geometry and a material, which are then added to the SCENE object for rendering. I used an Object to store the initial meshes, then added them to an Object3D; I later refined my approach once I understood how to properly employ the different container elements.
The basic wireframe approach involved creating a SCENE, CAMERA, and a renderer (which combines a SCENE and CAMERA to create what appears in the CANVAS element in HTML5). After that, I used a loop to create multiple MESHES using a Cube and a basic mesh material from ThreeJS. The last component was to create an animation cycle to move the group of boxes back and forth. This was done by putting a function to create movement inside of the render function (which behaves as a loop running at 60 FPS).
Once I understood the fundamentals of creating geometry and animation, I went back and made the cubes solid. This required using a Lambert material for the mesh with flat shading, in contrast to the basic material with wireframe. To complete the scene I needed some light for the material to receive so that the cubes would be visible, so I created some directional lights which got me familiar with those elements.
This exercise was an extension of the shaded cubes, where I wanted to see just how much I could push the capabilities of the webGL engine. I brought the size of the cubes down dramatically and increased the creation loop so that 10,000 cubes would be generated. The goal was to get an sense of the number and size required in order to create a dense field of objects. I decided to use small cubes since they would be easier to render than spheres, and because the size of the elements I wanted in the scene didn't really justify using spheres. Make the cubes small enough and the casual eye would not be able to tell the difference.
I decided to combine the bouncing between boundaries action of the wireframe exercise with the random star field to help me get an idea of the change in performance if I animated a large number of objects in the scene at once. The results were somewhat predictable; despite the simplicity of the objects in the scene, having thousands of elements to animate sapped the processing strength out of my older processor. This experience made me somewhat wary of using ThreeJS for incredibly complex concepts I had since there would undoubtedly be limits from a performance standpoint on who could enjoy it, in addition to any incompatibilities with browser support.
View the boundaries demoEver since I stopped needing to use the screen saver program After Dark I had missed the warp speed screen saver and decided that ThreeJS would provide a good opportunity to recreate it and learn about how to manage a scene. Using the star field boundary example I changed it so that the star field would come towards the viewer and then disappear. The entire star field was generated, rendered and animated in one go. The downside of this approach became apparent quite quickly so I set about rewriting the code to continuously generate stars and animate them.
The distribution of the stars during the animation ended up being uneven, which I set out to address when recreating the scene. Since the entire star field was rendered and then animated as one unit, the scene would have motion but lack a sense of dynamics. Before I decided to implement a continuous random star generator I toyed with the idea of creating the warp speed effect by creating star field groups one after the other. The problem with that approach is that having to wait for an entire group to clear the camera before it was disposed of would inflict a big performance drag. Just as one should endeavour to write as little code as possible to accomplish a task, I realized that as few objects as possible needed to be rendered in the scene.
My improvement to the warp speed prototype was to make a new creation function that contained the loop for generating stars. I also finally dispensed with the unnecessary intermediate step of storing the references to the meshes in an Object and then adding that to an Object3D. Now I had a function that would generate ten stars randomly every time it was called. I placed this function call in the render loop, resulting in about 600 stars being randomly generated a second. I disposed of unwanted stars that had moved out of frame by setting a cutoff point on the z-axis, after which a star would be removed from the Object3D being rendered.
The resulting scene ended up being a lot less dense than my previous examples, but an ideal warp speed simulation with a continuous stream of stars. There is a distinct twinkling in the center of the simulation, a result of the distant box that the stars are created within. This could be reduced by moving the area where the stars are born farther back from the camera, but this would have a downside since there would be more objects in the scene at one time causing performance drag.
Once I had mastered how to animate, generate, and dispose of objects properly I adapted the warp speed simulation to function as a background for a rail space shooter where a ship 'moves' by having the background move from right to left. This required altering the code to randomly create stars in an imaginary box to the right of the camera. The disposal line was also changed to slightly off camera left.
devtool: 'source-map'
webpack --watch
python -m SimpleHTTPServer 8080
updateUserName(name){this.setState({"user": name.target.value});}<input type="text" onChange={this.updateUserName}/>
{test: /\.less$/,loader: "style!css!less"}
loader:ExtractTextPlugin.extract("style-loader","css-loader!less-loader"),
{test: /\.less$/,exclude: /node_modules/,loader: ExtractTextPlugin.extract("style-loader?sourceMap", "css-loader? sourceMap!autoprefixer? browsers=last 5 version!less-loader?sourceMap"),},