Category Archives: Programming

How to Serve an Angular Single Page Application using Django – Part 3

Here’s part 1 of my series of how to serve an Angular 6 SPA web application in Django, without modifying the Angular CLI generated html. 

Putting it all in a Docker Container

I use Docker for development projects as it gives me a clean development environment where I can have all my dependancies isolated from other projects and stuff running on my system. 

Disclaimer: this may not be the best way to do Django and Angular in Docker. I chose this method because I wanted to have my Angular app served by Django to avoid CORS problems and just keep the architecture as close to what it’ll be in production as possible. Obviously it wouldn’t be running on Django’s runserver in production however. 

Here’s how to build a docker configuration to run my Django frontend and Angular backend together in the one container. 

First create a docker-compose.yml file:

version: '3'

services:
  db:
    image: postgres
  web:
    build:
      context: .
      dockerfile: Dockerfile
    working_dir: /code
    env_file:
      - web_variables.env
    command: sh devservers.sh
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

This docker-compose file first creates a postgresql database container and then a web container using a Dockerfile. When the container is started it runs the devservers.sh script we created in Part 2.

It mounts the current directory(the django root) to /code within the container. It then exposes port 8000 inside the container to port 8000 outside the container. 

In the Dockerfile put:

FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash -
RUN apt install nodejs
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
RUN mkdir -p /code/static
WORKDIR /code/frontend
RUN npm install -g @angular/cli
RUN npm install
RUN ng build --outputPath=/code/static

This creates a new Docker container from the Python 3 official image. It creates a /code directory in the container.

It then installs node.js as as dependancy for the Angular CLI and installs all the Django project dependancies from the requirements.txt file. 

Finally to test that the environment is ready it copies in the code from the Django root, installs the Angular CLI globally, installs the Angular project dependancies and does a test build of them.

Note that in the docker-compose, we’re telling it to mount the current working directory as a volume on /code. So the /code path in the container will be replaced with the Django root from the host system. 

Really this means that the lines from ADD onwards in the Dockerfile are unnecessary. However I’ve left them in there to just ensure that that we know about it at image build time if the environment won’t be ready to use. 

Using the Docker environment

To start the dev servers run the following command in the Django project root:

docker-compose up -d

As your Django project root is mounted in the Docker container, any changes you make to the Django/Angular project will also be in /code in the Docker container. The Django manage.py runserver will automatically detect any changes on the Django side and recompile and serve them.

Angular build watch will detect changes to the Angular code in <Django project root>/frontend and rebuild them, putting the output SPA into /code/static where it’s served by Django’s static file serving. 

If you add new files or npm dependancies to the Angular project, you’ll need to restart the web container using:

docker-compose restart web

If you add new dependancies to the Django project do a full rebuild with:

docker-compose up --build -d 

In my case I actually run my dev install on a different machine than the one I write code on, so I just upload via Webstorm or PyCharm to the project root on my dev server. The changes are detected and ready to use within seconds. 

How to Serve an Angular Single Page Application using Django – Part 1

Here’s part 1 of my series of how to serve an Angular 6 SPA web application in Django, without modifying the Angular CLI generated html. 

Configuring Django to Serve an Angular SPA

I’ve been working on a blog/gallery web application that has is using Django to provide a RESTful backend API and has an Angular 6 front end client side. In doing this I wanted to have the Angular single page application served by the Django server. Configuring this was a little complicated as there was no clear complete explanation.

Other tutorials do exist, but they require modifying the files generated by the Angular CLI to fit into Django’s templating and static file system. See this for an example: https://medium.com/swlh/django-angular-4-a-powerful-web-application-60b6fb39ef34

I didn’t want to have to do this. Using Whitenoise and Django SPA, you can serve the Angular application without modification. 

Add the dependancies to the requirements.txt:

whitenoise
django-spa

Add whitenoise to the INSTALLED APPS list in settings.py:

INSTALLED_APPS = [
    'whitenoise.runserver_nostatic',
    'django.contrib.staticfiles',
    ...
]

Configure static files in settings.py:

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.11/howto/static-files/

STATIC_URL = '/static/'
STATICFILES_STORAGE = 'spa.storage.SPAStaticFilesStorage'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')

When deploying your web application, create a “static” directory under the Django project base. Use “ng build” to build your Angular web application, then copy everything from the “dist/<your_angular_app>/” directory to the “static” directory in Angular. 

After starting the Django server:

python3 manage.py runserver 0.0.0.0:8000

you should see your Angular SPA start up and run at http://<server>:<port>/.  You’ll also be able to access the application via http://<server>:<port>/static/, however all the urls in there will reference /, so it’s best to use the root url.

You can see a full example of this in the pnb gallery project on GitHub.

In the next part I’ll show you how to get the two dev servers running together to autobuild both Django and Angular while having the Angular SPA served by Django. 

Speed up Deploy via SCP in GitLab CI/CD

Deploying thousands of small files via SCP to a server takes an inordinately long time, even via a very fast network, much longer than transferring one large file. Here’s a tutorial on my Gitlab CI setup for compressing all my deployment files into one large tarball, transferring it to to the server then uncompressing it there. 

I have been experimenting with GitLab CI/CD for use with my Swimming Management System projects for Masters Swimming Queensland. It’s a legacy project that I’m now gradually transitioning to modern standards. I’ve set up a Pipeline that will allow me to test commits in PHPUnit, then have them automatically deploy to a dev/test server. 

The project now uses both Composer and with some Angular modules it also uses NPM. In the build phase on GitLab CI, composer install is run to get the dependancies into the vendor directory. 

To do it this way, you’ll need to have shell access via ssh to your server. 

I’ve set up the test server deployment details and authentication credentials as in GitLab Variables:

Configuration of GitLab CI/CD Variables
Configuration of GitLab CI/CD Variables

In the deploy stage I’ve added the following code prior to upload via scp:

- tar -czf /tmp/build.tar.gz .
- echo "TEST_SSHPATH=${TEST_SSHPATH}" > sshenv
- sshpass -e scp -P ${TEST_SSHPORT} -r -o stricthostkeychecking=no sshenv ${TEST_SSHUSER}@${TEST_SSHHOST}:~/.ssh/environment

In this case, the target ‘.’ indicates that we are tarballing the current directory.

In the second line we create a file that contains the definition of an environment variable on the target server, which is set to the GitLab CI variable TEST_SSHPATH. TEST_SSHPATH contains the path to the root of my code deployment on the server.

Finally, we then scp this file to the target server, putting it in ~/.ssh/environment. This will mean that when we ssh into the server, that environment variable will be available to us, containing the value from the GitLab CI variable. 

Now we can transfer the build.tar.gz file and un-tar it. 

- sshpass -e scp -P ${TEST_SSHPORT} -r -o stricthostkeychecking=no /tmp/build.tar.gz ${TEST_SSHUSER}@${TEST_SSHHOST}:${TEST_SSHPATH}
- sshpass -e ssh -p ${TEST_SSHPORT} -o stricthostkeychecking=no ${TEST_SSHUSER}@${TEST_SSHHOST} "cd ${TEST_SSHPATH}; tar -zxf build.tar.gz"

The first line in this block does the transfer via scp, to our the path provided in the variable TEST_SSHPATH.

The second line, connects via ssh , changes directory to the code deployment path, then extracts the build files. Tar -zxvf removes the .tar.gz file after it’s done, so there’s no need to seperately delete it.

Before making this change, my swimman project would take 16:30 to build and install. With this change it’s down to 4:35. That’s a saving of 12 minutes which makes a big difference when deploying a quick fix to the test environment. 

Monitoring MQTT Services in an Angular Web Application

I’m going to show you how to use the Paho MQTT JavaScript implementation in an Angular application to subscribe to a topic and display the message content in a web browser.

I’ve built this as part of my ongoing project to build a custom Weather Station for my home. 

Prerequisites:

  • An Angular CLI project
  • A working MQTT Broker to access with accessible topics that are receiving messages

MQTT Packages for Angular

There’s a few MQTT related packages for Angular integration on npm. I’ve chosen to use ng2-mqtt. It works for me at this point. I did consider using ngx-mqtt, but as it’s build was marked as failing at the time of writing(13/11/2018) I decided to use the simpler ng2-mqtt.

Procedure

First, set up an Angular CLI Project. I’m using Angular 7.0.3. 

Install ng2-mqtt using npm:

npm install --save ng2-mqtt

For simplicity of testing and demonstration, I’m using ng2-mqtt in my AppComponent. As I further develop my weather station display app, I’ll probably move it to a specific service for managing weather data.

Import Paho at the top of your app.component.ts:

import {Paho} from 'ng2-mqtt/mqttws31';

Add some properties for variables to the class AppComponent for storing the incoming data. Change these to match your needs:

windSpeed: Number;
windDirection: Number;

Also add a private member for the MQTT client. I’ve also put in a variable for storing the IP address or hostname of my MQTT broker. In a real world implementation you’d put this in some sort of configuration management.

private client;

mqttbroker = 'localhost';

Your AppComponent should implement OnInit:

export class AppComponent implements OnInit {

Then add a ngOnInit implementation:

ngOnInit() {
  this.client = new Paho.MQTT.Client(this.mqttbroker, Number(9001), 'wxview');
  this.client.onMessageArrived = this.onMessageArrived.bind(this);
  this.client.onConnectionLost = this.onConnectionLost.bind(this);
  this.client.connect({onSuccess: this.onConnect.bind(this)});
}

In this code we create a client object and tell it the address of the MQTT Broker and the port number we’re using. I’ve given my client the name ‘wxview’. We then specify callbacks that will be used for when a message is received, a connection is lost, and a connection is established.

Note when setting references to member functions that handle each event, we call the .bind(this) function. This ensures that “this” in the callback function refers to the AppComponent class, not the MQTT Client. For more information, see the Function.prototype.bind() article on Mozilla Developer Network.

Now we can set up the onConnect function:

onConnect() {
  console.log('onConnect');
  this.client.subscribe('wxstation/wind_speed');
  this.client.subscribe('wxstation/wind_direction');
}

This function tells the client to subscribe to specified topics. In my case I’ve used topics that are specific to my weather station project. You can change the topics to be subscribed to for your project specific requirements.

We set up onConnectionLost as per the documentation:

onConnectionLost(responseObject) {
  if (responseObject.errorCode !== 0) {
    console.log('onConnectionLost:' + responseObject.errorMessage);
  }
}

This will just log the error to the browser console if the connection is lost.

Then we build our function to handle any messages received:

onMessageArrived(message) {
  console.log('onMessageArrived: ' + message.destinationName + ': ' + message.payloadString);

  if (message.destinationName.indexOf('wind_speed') !== -1) {
    this.windSpeed = Number(message.payloadString);
  }

  if (message.destinationName.indexOf('wind_direction') !== -1) {
    this.windDirection = Number(message.payloadString);
  }

}

In this function I first log the message topic(message.destinationName) and message(message.payloadString) to the web browser console. 

I then check the topic of the message received and update the appropriate member variable.

To finally put this all together, build a template that binds to these variables:

<div class="container-fluid">
  Wind Speed: {{windSpeed}}<br />
  Wind Direction: {{windDirection}}
</div>

Now when you open your app in the browser, you’ll see the web page continually update as new MQTT messages are received by the broker. 

The Full Code

Check it out on Stackblitz.

Study Notes for Java SE 7 Programer II

Here are my study notes for the Java SE 7 Programmer II to get the Oracle Certified Professional, Java SE 7 Programmer certification. This is based on the topics in Oracle published syllabus for the exam.

This page is a work in progress, so come back for more details as I go.

See also my Study Notes for Java SE 7 Programmer I OCA.

Note: I accept no liability for any inaccuracy or mistakes in this document. If you find any errors, please let me know so I can correct it!

Java Class Design

  • Use access modifiers:
    • private
      • only accessible within class
    • protected
      • only accessible with package and subclasses
    • public
      • world accessible
    • default
      • package-private
      • accessible within package
  • Override methods
  • Overload constructors and methods
    • Overloaded method use the number and types of arguments to determine which version of the overloaded method to call
    • Return type are not used to determine which overloaded method to call.
    • If a class will be inherited, a no argument constructor should be supplied. This will be called when a subclass is instantiated.
  • instanceof operator
    • Comparison between object and specified type.
    • Will return true when an object is:
      • An instance of the specified class
      • An instance of a subclass of the specified class
      • An instance of a class that implements the specified interface
    • Will return false when an object is:
      • An instance of a parent class of the specified class
      • A reference to null
    • Will not compile if comparing neither a subtype or supertype
      • Will produce compilation error: Unresolved compilation problem: Incompatible conditional operand types.
    • Often over-used and can be replaced by polymorphism
  • Casting
    • Used with superclass reference types allow access to subclass properties and methods
    • Uses syntax (MyClass)myInstance.
  • Virtual method invocation
    • Used when multiple subclasses override the same method
    • The object type’s implementation of the overriden method is used, not the reference type method
    • Important part of polymorphism
  • Override methods from Object class:
    • hashCode
      • Must consistently return the same integer for the same object each time invoked, provided information used by equals method is not changed.
      • If two objects are not equal they should not return the same integer value
    • equals
      • Compares two objects
      • Must be reflexive, symmetric, transitive and consistent
      • Comparing to null should return false
    • toString
      • Must return a string that represents the object as text
      • Recommended that all subclasses override this method
  • package statement
    • The package statement must be the first executable line of the source file, preceding any import statements.
    • Creates a package, put package statement with the name of the package at the top of every source file that contains types(classes, interfaces, enumerations and annotations) to be part of the package.
    • Without a package statement the type becomes part of an unnamed package.
  • import statement
    • Import a package member using import package.Member;
    • Import an entire package by using import package.*;
      • Packages are not hierarchical, importing package.* will not import package.subpackage.*
      • Where members of two or more imported packages have the same name, the fully qualified member name must be used.
    • import static statement allows you to reference constants and static methods from a class without prefixing the class name.

Advanced Class Design

  • Abstract classes
    • Cannot be instantiated but can be subclassed
    • May or may not include abstract methods
    • Subclass must provide implementations for all abstract methods of parent, otherwise must also be abstract.
  • static keyword
    • Static methods must use object reference to access instance variables or methods.
    • Static methods cannot use this keyword.
    • Static methods can be hidden by a static method with the same signature
      • Differs from overriding because the version of the static method that is invoked depends on the reference type from which it is invoked
    • static keyword is used to declare a class variable, which is part of the class not the object instance
    • Class variables are shared between all instances of the class
  • final keyword
  • Create top-level classes
    • Top level classes expected to be inherited by subclasses should have a default no-arg constructor.
    • The default no-arg constructor cannot be used if the class has any constructors defined.
  • Create nested classes
    • Static Nested classes
      • Declared using static keyword
      • Do not have access to members of enclosing class
    • Non-static nested classes
      • Known as inner classes
      • Have access to members of enclosing class, including private members
      • Cannot define any static members
    • May be private, public, protected or package-private
  • Enumerated types
    • Allows a variable to be set equal to one of a set of predefined constant values.
    • Is a class which extends java.lang.Enum.
    • Constructor must be package-private or private.
    • Java compiler adds a values() method at compile time which returns an array of values in the enum.

Object-Oriented Design Principles

  • Interfaces
    • Declaring
    • Implementing
    • Extending
  • Interface inheritance or class inheritance
    • Multiple interfaces can be implemented
    • Multiple unrelated classes can inherit interfaces.
  • Applying
    • Cohesion
      • The degree to which the elements of a class belong together
      • Cohesion in a class is increased if the functions within a class have a lot in common and perform a small number of related tasks.
      • Advantages:
        • Reduced complexity
        • Increased maintainability
        • Increased reusability
    • Low-coupling
      • The degree to which each component uses and has knowledge of other components.
      • Advantages:
        • Components from a low-coupling system can be replaced with alternative implementations that provide the same functions.
        • Components may be implemented with different platforms, using different languages, operating systems or build environments.
    • IS-A
      • Can be identified by a class using the extends or implements keywords.
    • HAS-A relationship
      • A class uses HAS-A relationship by composition, by having an instance of another class and using it.
  • Object composition principles
  • Singleton design pattern
    • Only one instance of the object is created.
    • Possible implementations:
  • Data Access Object(DAO) design pattern
    • Consists of:
      • A Data Access Object Interface
      • A Data Access Object concrete class that implements the DAO interface
      • A Model object, containing get/set methods for the data
  • Factory pattern
    • Consists of:
      • 1 or more concrete classes that implement a common interface
      • A common interface
      • A factory class which returns a new instance of one of the concrete classes

 

Generics and Collections

  • Generics allow the creation of classes, interfaces and methods where the type is specified as a parameter.
  • Generic classes
    • Declared using <> symbols
  • Diamond for type inference
    • Means that the new statement to instantiate the object and call the constructor doesn’t require repetition of the type(s) from the reference declaration.
    • The compiler infers the type from the declaration.
    • e.g. ArrayList<string> myList = new ArrayList<>();
  • Raw types
    • A raw type is a generic class or interface without a type parameter supplied
    • A parameterized generic type may be assigned to it’s raw type
    • Assigning a raw type to a parameterized type will produce a warning
    • Using a raw type to invoke generic methods will produce a warning
  • Collections using raw and generic types
  • Wildcards
  • Wrapper classes
    • The type argument of a generic cannot be a primative type, they must be object types
    • Always declare generics using object types.
    • Using a primative type will cause a compile error
  • Autoboxing
    • Autoboxing will automatically convert a primative variable to the corresponding type wrapper.
    • e.g. an int value can be assigned to ArrayList<Integer> myVals
  • Unboxing
    • Unboxing means that a type wrapper object value can be automatically converted to a primative value when assigning a value copied from the type wrapper object.
  • Implementations of:
    • List
    • Set
    • Deque
    • Map
      • Sorted Maps
        • TreeMap
        • NavigableMap
      • HashTable
      • HashMap
  • Use java.util.Comparator
  • Use java.lang.Comparable
    • Must provide a method int compareTo(T o) which returns -1,  or 1.
  • Sort and search arrays and lists
    • To be sorted in collections, the objects contained within must implement the Comparable interface.

String Processing

  • Scanner
    • Constructor may take File, InputStream, Path, Readable, ReadableBytChannel and String
    • Can parse text into primitive types and strings using regular expressions
    • useDelimiter method accepts a regular expression as a Pattern or a String to use as a delimiter between fields in the text
    • has and next functions exist for each primitive type and some others
    • nextLine advances to the next text line and returns the current one as a String.
  • StringTokenizer
    • Constructors:
      • StringTokenizer(String str) – uses default delimiters – space, tab, newline, carriage return, form feed.
      • StringTokenizer(String str, String delim) – Uses specified delimiter in delim
      • StringTokenizer(String str, String delim, boolean returnDelims) – if returnDelims is true, include the delimiters in the tokens
    • nextElement() returns the next token
    • nextToken() returns the next token or thorws NoSuchElementException
    • countToken() calculates the number of times nextToken can be called before an exception
    • hasMoreTokens() returns true if there is at least one more token after current token
  • StringBuilder
    • Constructors:
      • StringBuilder() – creates a blank String Builder
      • StringBuilder(CharSequence seq) – creates a StringBuilder with the same characters as the seq
      • StringBuilder(int capacity) creates a StringBuilder with capacity set
      • StringBuilder(String str) creates a StringBuilder with the contents of the String
    • append(T x) methods exist for all primatives, CharSequence, String and StringBuffer
    • charAt(int index) returns a char from the specified index
    • delete(int start, int end) deletes characters from start to end.
    • insert(int offset, T x) methods exist to insert primatives at an offset
    • substring(int start) returns a String containing all characters from start to the end of the StringBuilder
    • substring(int start, int end) returns a String containing characters from start to end.
    • toString() returns the StringBuiler as a string.
  • String
  • Formatter
    • Interprets printf style format strings
    • Example: Formatter formatter = new Formatter(sb, Locale.US);
  • String processing with regular expressions
    • Used by Pattern and Matcher classes.
    • PatternSyntaxException indicates that a syntax error has occurred in a regular expression pattern.
    • Syntax:
      • ^ – matches beginning of line
      • $ – matches end of line
      • . – matches any single character except newline
      • […] – matches any single character in brackets
      • [^…]- matches any single character not in brackets
      • \A – Beginning of entire string
      • \z – End of entire string
      • \Z – End of entire string except final line terminator
      • a|b – matches either a or b
      • \w – matches word characters
      • \W – matches nonword characters
      • \s – matches whitespace
      • \S – matches non whitespace
      • \d – matches digits, same as [0-9]
      • \D – matches nondigits
      • \n – matches newlines
      • \t – matches tabs
  • Format strings
    • See Formatter javadoc
    • %n is used for platform appropriate new line
    • %d is used for a decimal integer
    • %f is use for float
    • a leading 0 after %d indicates use of leading zeroes
    • a number flag indicates characters in width, e.g. 10.3 is ten characters wide with three places after decimal point
    • + indicates use sign, positive or negative
    • , indicates locale specific grouping characters
    • – indicates left justified
    • defaults to right justified
    • To use a % sign in a formatted number for instance for a percentage, use %%.

Exceptions and Assertions

  • Each try statement requires at least one catch or a finally block.
  • catch statement
    • Once one catch statement executes the otheres are bypassed.
    • Exception subclasses must come before their superclasses. Otherwise an unreachable code error will occur at compile time.
  • throw statement
    • thrown object must be of type Thowable or a subclass of it.
    • Execution immediately stops and control is passed to the nearest matching catch statement.
  • throws statement
    • Lists the exception types the method may throw.
    • Required for all thrown exceptions except Error or RuntimeException and their subclasses.
  • Multiple Exception types in single catch block
    • Called multi-catch
    • Use pipe symbol
    • e.g. catch(FileNotFoundException | IOException e)
  • try-with-resources statements
  • finally statement
    • Executes regardless of whether or not an exceptions has been thrown.
    • If exception has been thrown executes after the relevant catch block.
  • Custom exceptions
  • Test invariants by using assertions

Java I/O Fundamentals

  • Read and write data from the console
    • Get a Console object with System.console() method.
    • System.console().readPassword() retrieves user input without echoing typed characters.
    • The Java platform supports three standard streams:
      • Standard input – stdin – using System.in
      • Standard output – stdout – using System.out
        • System.setOut may be use to redirect output to a different stream
      • Standard error – stderr – using System.err
    • Can be read using BufferedReader or Scanner
  • Using streams to read and write files with:
    • BufferedReader
      • Implements Closeable, AutoCloseable, Readable
      • read() reads a single character – returns -1 if end of stream is reached
      • ready() returns true or false
      • skip(long n) skip n characters
      • Methods throw IOException
    • BufferedWriter
    • File
    • FileReader
    • FileWriter
      • Implements Closeable, Flushable, Appendable, AutoCloseable
      • Defaults to overwrite an existing file use FileWriter(File file, boolean append) constructor with append set to true to append to an existing file.
      • Inherits methods from Writer.
    • RandomAccessFile
      • Implements DataOutput, DataInput, Closeable
      • readUTF reads the file into a String
      • writeUTF writes a String to the file
      • Maintains a cursor also known as a current file pointer marking the current location in the file.
      • seek(long pos) sets the cursor location/file pointer, measured in bytes from beginning of file.
      • read and write methods are provided for all primatives
      • readLine reads bytes from the file, starting from current file pointer and ending at the end of the current line. Line terminating characters are not included.
    • FileInputStream
      • Implements Closeable, AutoCloseable
      • read() reads a single byte or -1 if end of file
      • read(byte[] b]) reads b.length bytes of data into an array of bytes
      • read(byte[] b, int off, int len) reads len bytes starting at offset off into an array of bytes
      • skip(long n) skips n bytes
    • DataInputStream
    • DataOutputStream
    • ObjectOutputStream
    • ObjectInputStream
    • PrintWriter
  • Serializable interface
    • Enables a class to be serialized
    • Always include a serialVersionUID field
    • static and transient fields are not serialized
    • Superclass fields are only serialized if the superclass also implements Serializable

Java File I/O (NIO.2)

  • Path class
    • Paths.get returns a Path object
    • resolve(Path p) appends a relative path p to this path. If p is an absolute path then p is returned.
    • relativize(Path p) returns a relative path from this path to path p
    • getName(int index) returns the name of the component of the path at that index
    • getRoot() returns root component
  • Files class
    • Atomic operations
      • An atomic file operation cannot be interrupted or partially completed.
      • Entire operation is completed or fails.
    • move method supports atomic moves and replacing existing files by means of options provided
  • Interfaces:
    • BasicFileAttributes
      • creationTime() returns creation time as a FileTime object
      • fileKey() returns an object that uniquely identifies the file
      • isDirectory() returns true if the file is a director, false otherwise
      • isOther() returns true if the file not a regular file, directory or symbolic link
      • isRegularFile() returns true if the file is a regular file
      • isSymbolicLink() returns true if the file is a symbolic link
      • lastAccessTime() returns the last access time as a FileTime object
      • lastModifiedTime() returns the last modification time as a FileTime object
    • BasicFileAttributeView
      • readAttributes() method returns a BasicFileAttributes object
      • setTimes() allows any or all of a file’s last modified time, last access time and create times to be updated.
    • DosFileAttributes
      • Provides ability to read legacy MS DOS file attributes
      • Provides isArchive(), isHidden(), isReadOnly, isSystem() attributes
    • DosFileAttributeView
      • Provides readAttributes which returns a DosFileAttributes object
      • Provides setArchive(boolean value), setHidden(boolean value), setReadOnly(boolean value), setSystem(boolean value)
    • PosixFileAttributes
      • Provides ability to read file attributes from POSIX operating system file systems. Typically these are Unix, Linux or Mac OS X systems.
    • DirectoryStream
    • FileVisitor
    • PathMatcher
      • Instantiate with for example:
        • PathMatcher matcher = Filesystems.getDefault().getPathMatcher(“glob:*.{txt,doc}”);
      • Uses glob patterns
    • WatchService
      • Watches registered objects for changes and events.
      • May be used to monitor directories or files for changes.
      • If the file system does not support file change notifications the API will poll the file system for changes.

Building Database Applications with JDBC

  • JDBC Drivers must include Statement, ResultSet and Connection objects
  • JDBC Driver Types:
    • Type 1
      • Not pure Java
      • Require native libraries
      • Use bridging to data source
    • Type 4
      • Are pure Java
      • Use direct connections to data source
      • Can use network protocols
  • Core interfaces:
    • Driver
      • acceptsURL(String url) method checks whether the driver thinks it can open a connection to the url.
      • connect(String url) is the most basic way to start a connection
      • connect(String url, Properties info) method tries to start a connection
      • connect(String url, String user, String password) also tries to start a connection
      • url is in format: jdbc:subprotocol:subname
      • getMajorVersion() and getMinorVersion() get the major and minor version numbers respectively
      • SQLException is thrown for most database access errors.
      • See Javadoc for other methods.
    • Connection
      • Connections are AutoCloseable
      • Use DriverManager.getConnection() and DataSource.getConnection() to establish a connection using a JDBC driver
      • Use setAutoCommit(false) to turn off auto commiting of updates
      • Use commit() to commit updates when auto committing is off.
    • Statement
      • Statements default to being commited automatically
    • ResultSet
      • Maintains a pointer/cursor to the current row.
      • next method moves to next row, but returns false if there a no more rows.
      • Current row pointer is initially set before the first row, so will require a next to reach the first row.
      • By default is only cursor only moves forward
      • By default ResultSet is not updateable
      • A ResultSet may be set to be updateable and scrollable.
  • Connect to database using DriverManager class
  • Submit queries and read results
  • Use JDBC transactions
    • Disable auto-commit mode using setAutoCommit(false);
    • Commit changes using commit();
    • Rollback changes rollback() if a SQLException is thrown.
    • setSavepoint() sets a Savepoint within the current transaction. Use rollback(Savepoint s) to rollback to that savepoint.
    • Disable auto-commit only during transaction mode.
    • Savepoint interface
      • Represents a point that can be returned to by the Connection.rollback method.
      • releaseSavepoint(Savepoint s) removes the Savepoint from the current transaction.
  • Construct and Use RowSet objects with RowSetProvider class and RowSetFactory interface
    • Get a RowSetFactory using RowSetFactory factory = RowSetProvider.newFactory();
    • Get a JdbcRowSet using JdbcRowSet rs = factory.createJdbcRowSet();
    • A JdbcRowSet can also be obtained by using JdbcRowSet rs = new JdbcRowSetImpl();
  • CachedRowSet interfaces
    • These are disconnected RowSets – connected only when reading data and propagating changes to the data source
    • WebRowSet interface
      • Allows a RowSet to be read or written in XML format.
    • FilteredRowSet interface
      • Allows disconnected filtering of a RowSet using a Predicate object
  • Statement object
    • Supports only simple SQL statements without parameters
    • Support only a single result set open
  • PreparedStatement object
    • Supports only input parameters
  • CallableStatement object
    • Supports input and output parameters because they contain stored procedures

Threads

  • Create and use Thread class and Runnable interface
  • Manage and control thread lifecycle
    • Threads may be paused in execution using Thead.sleep(long milliseconds) which throws an InterruptedException
    • isAlive() method returns true if the thread is still running
    • join() method waits for a thread to finish execution
  • Synchronise thread access to shared data
  • Identify code that may not execute correctly in a multi-threaded environment

Concurrency

  • Collections from java.util.concurrent package
    • BlockingQueue provides a first-in-first-out queue data structure
    • ConcurrentMap interface has atomic operations to remove or replace a key-value pair if it’s present or add a key-value pair if it is absent. Has implementation ConcurrentHashMap.
      • ConcurrentHashMap is similar to HashMap
      • ConcurrentNavigableMap is sub-interface supporting approximate matches.
        • Standard implementation is ConcurrentSkipListMap which is similar to TreeMap.
    • These collections try to avoid Memory Consistency Errors using a happens-before relationship.
  • Use
    • Lock interface
      • lock() acquires the lock
      • unlock() releases the lock
    • ReadWriteLock interface
      • Separate locks for reading and writing
      • Read lock may be held by multiple reader threads at the same time if there are no writers
      • Write lock may only be held by one thread
    • ReentrantLock
      • Mutual exclusion lock – both reading and writing are locked.
      • lock() returns immediately if the lock is acquired, disables thread and waits if lock is not available.
      • Optional constructor Boolean fairness parameter, when set to true locks favour the longest-waiting thread.
    • Executor interface
      • execute(Runnable r) method creates a thread of Runnable object r.
      • May use a worker thread or put Runnable object r in a queue waiting for a worker thread.
    • ExecutorService subinterface of Executor
      • Provides features to manage lifecycle of tasks and Executor
    • Executors
      • Factory and utility methods for Executor, ExecutorService, ScheduledExecutorService, ThreadFactory, Callable.
      • Provides methods for creating a thread pool: newCachedThreadPool, newFixedThreadPool, newScheduledThreadPool, newSingleThreadExecutor, newSinglethreadScheduledExecutor, newWorkStealingPool.
      • Allow separation of thread management and creation from the rest of application.
    • Callable interface
      • Implementations must define a method call with no arguments that returns a result of a generic type. May thrown an exception if unable to produce a result.
    • Future interface
      • Represents the result of an asynchronous computation.
      • Provides method to check for completion, to wait and to retrieve results.
  • Use parallel Fork/Join Framework
    • Implementation of ExecutorService interface
    • Uses work-stealing algorithm
    • Extend ForkJoinTask or a more specialised subclass such as RecursiveTask or RecursiveAction
    • RecursiveTask returns a result.
    • RecursiveAction does not return a result.

Localisation

  • Read and set locale with Locale object
    • Use Locale constructors
    • Use constants in Locale class
    • Use Locale.forLangaugeTag factory method
    • Locale.Builder class
      • Create a Locale variable with the Locale.Builder class: Locale loc = Locale.Builder().setLangauge(“en”).setScript(“Latn”).setRegion(“AU”).build();
      • build method must be called to create a Locale object withtthe Locale.Builder class.
  • Build a resource bundle for each locale
    • Resource bundles stored in properties files using name=value pairs
    • Properties files must be in the CLASSPATH
    • There must be an individual properties file for each language supported
  • Call a resource bundle from an application
    • Identified by family name/base name and may have a two character language code.
    • e.g. Sample_en_AU where en is the language code and AU is the country code for Australia
    • Create a local object for the language, country and platform appropriate
    • Load bundle using
      • Locale myLocale = new Locale(“fr”, “CA”, “UNIX”)
      • ResourceBundle myBundle = ResourceBundle.getBundle(“Sample”, myLocale);
      • will search for classes in this order:
        1. Sample_fr_CA_UNIX
        2. Sample_fr_CA
        3. Sample_fr
        4. Sample_en_US
        5. Sample_en
        6. Sample
    • Resource bundle may be stored in classes extending ListResourceBundle, PropertyResourceBundle or a properties file.
  • Format dates, numbers and currency values with NumberFormat and DateFormat classes
    • Use DateFormat myDateFormat = DateFormat.getDateInstance(DateFormat.DEFAULT, Locale.US); to format a date to US format.
  • Advantages of localising an application
  • Define a locale with language and country codes

My Study Notes for Java SE 7 Programmer I(OCA) Exam

After completing the Java SE 7 Programmer I(OCA) Exam Prep Seminar, I took a self-test online to test my knowledge. Based on my results, particularly the mistakes I made, here are my notes on things you definitely need to know to pass the Java SE 7 Programmer I exam. See my preparation for Java SE 7 Programer I(OCA) exam article for more details on how I prepared.

Firstly, check you understand the assumptions published in the Exam Topics page provided by Oracle.

This page is not intended to teach you Java or enable you to cram for the exam. It makes the assumption you’re already a Java programmer and just need to brush up on a few things you’d normally use documentation for. It’s about having a list of important things to know in the exam.

Note: I accept no liability for any inaccuracy or mistakes in this document. If you find any errors, please let me know so I can correct it!

Java Basics

  • Variable scope
    • Class scope
      • Non-static variables defined in a class are accessible within the class methods without using the class or instance name or “this”.
      • Static variables
    • Method scope
      • Variables declared in the method and arguments of the method are only available within the method.
      • If an argument of the method has the same name as a class variable, the method’s version will be used rather than the class version.
      • If a variable with the same name as a class variable is declared in the method, the method’s version will be used rather than the class version.
    • Block scope
      • A variable declared in a block of code will only be available in that block of code.
      • Variables may be declared anywhere in a block or method but will only be valid after the declaration.
      • Variables declared in a block will be accessible inside a new nested scope under the scope they were declared in.
      • You cannot declare a variable in an inner block scope with the same name as a variable in the outer block scope.
    • Variables are created only when their scope is entered.
    • Variables are destroyed when their scope is left.
  • Class structure
  • Executable applications
    • A Java class is made executable from the command line by creating a main method using the declaration: public static void main(String args[])
      • A main method that returns a value will compile but will produce an error when executed
    • First command line argument is args[0].
  • Import Java packages
    • Import package member using import package.member
    • Import complete package using the wildcard character
    • Java compiler automatically imports the java.lang package and the current package
    • If more than one package you have imported contains the same member name, you must refer to it using the member’s fully qualified name.

Java Data Types

  • Declare and initialise variables
    • Declared using syntax: type identifier
    • Multiple variable of the same type may be declared in the same statement, separating the identifiers with commas.
    • Variables may be declared and initialised on the same line.
    • Multiple variables of the same type may be declared and initialised on the same line, by putting a comma after the value of each variable and then supplying the next identifier.
    • Variables may be dynamically initialised with any valid expression at time of declaration.
  • Object reference variables
    • When instantiating an instance of a class, we declare a reference variable and then allocate a new instance of the class to the reference
    • Class name and arguments specify the constructor to be used
  • Primitive variables
    • 8 Primitive types:
      • byte – 8 bits signed
      • short – 16 bits signed
      • int – 32 bits signed
      • long – 64 bits signed
      • float – 32 bits single precision
      • double – 64 bits double precision
      • char – 16 bits unicode
      • boolean – true or false – not 0 or 1
  • Integer literals:
    • Whole number value is an integer literal, assumed to be decimal
    • Octal values denoted by a leading 0
    • Hexadecimal values denoted by a leading 0x
    • Binary values denoted by leading 0b – beginning with JDK 7
    • No error produced when assigning an integer literal that is within the range of the type
    • To specify a long literal append an L(upper-case or lower-case) to the end of the literal
    • Underscores may be embedded in integer literals
  • Floating-point literals:
    • Default is double precision
    • To specify float, append F to the literal
    • To specify double, append D to the literal
    • Scientific notation uses a floating point number plus suffix indicated by E followed by a decimal number with is the power of 10 to which the number is multiplied.
    • Hexadecimal floating point literals may be used with prefix 0x, but scientific notation then uses P rather than E.
    • Underscores may be embedded in floating-point literals
  • Character literals:
    • Enclosed in single quotes(‘)
  • Read and write object fields
    • Inside the class just use the field name.
    • Outside the class use the instance name, dot operator and field name.
  • Object lifecycle
    • Variables are created when their scope is entered and destroyed when their scope is exited.
    • Objects will be eligible for garbage collection when there is no reference pointing to them.
  • Call object methods
    • Within the class just use the method name.
    • Outside the class use the instance name, dot operator, method name and required arguments.
  • StringBuilder class see tutorial
    • Identical to StringBuffer, is faster, but is not synchronized so is not thread-safe
    • Represents growable and modifiable character sequences
    • Provides utility functions for characters or substrings to be inserted into or appended onto a string.
    • Use append() to add more characters to the string
    • Use charAt() to get the character at a particular index – throws IndexOutOfBounds exception if index is invalid.
    • Use length() to get length of string
    • Use substring(int index) to return all characters from index to end of string or substring(int start, int end) to return characters between start and end inclusive.
  • Strings
    • Strings are immutable – cannot be changed once created
    • A string object is automatically created for every string literal
    • The + operator may be used to concatenate strings
    • To compare two strings use the equals() or equalsIgnoreCase() methods of the String class
    • Use the compareTo() or compareToIgnoreCase() method to compare whether a string comes before or after another string in dictionary order.
    • Use length() to get length of string
    • String methods that return a new String object may be used with the assignment operator to replace the an existing String object.
    • String methods that “modify the string” don’t actually modify the string, but return a modified copy of the String.

Operators and Decision Structures

  • Operators
  • Parentheses
    • Raises the precedence of operations inside them
    • May be used to alter the precedence or clarify the meaning of the expression
  • Object and string equality
    • == determines if the variable references refer to the same object
    • .equals compares the contents of the two objects
  • If/else statements
    • May contain just one statement or a block
    • May be nested, but the if-else-if ladder structure is preferred
  • Switch statements
    • Statements in a switch case must be labeled with one or more case or default labels.
    • break statement branches execution to first statement after switch block
    • break may be omitted from a case in order to have execution fall through to the next case’s statements
    • Prior to JDK 7 only byte, short, int or char expressions may be used in a switch.
    • In JDK 7 and above String may be used.
    • switch is more efficient than if’s, but may only test equality of contents of expression

Arrays

  • One-dimensional arrays
    • Braces may follow the type or the variable name.
    • Array memory must be allocated using the new keyword and size must be specified in braces after the type
  • Multi-dimensional arrays
    • Braces may follow the type or the variable name.
    • When specifying the size of a multi-dimensional array only the first dimension size needs to be specified. The second dimension size may be allocated manually for each index of the first dimension.
    • It is possible to create uneven or irregular multi-dimensional arrays – a triangular multi-dimensional array even!
  • java.util.Arrays has utility static methods for manipulating arrays
  • Arrays are objects and all methods of class Object may be invoked on an array.
  • ArrayList
    • Supports dynamic arrays which can grow when needed.
    • Automatically enlarged when needed.
    • May be shrunk when objects are removed.
    • Declared in form: ArrayList<type> var = new ArrayList<type>()
    • Contents may be returned using default toString() method.
    • Can be converted to an array using toArray() method. Needed for use with an enhanced-for loop.
    • Use get(index) method to access a value.
    • Use add(element) to append to list
    • Use add(index, element) to insert in a list – index must be less than the size() of the ArrayList.
    • Use remove(index) to remove from ArrayList.
    • Use remove(object) to remove the first occurrence of object from the ArrayList.
    • Use clear() to remove all elements from the arraylist.

Loops

  • While loops
    • Only run if controlling expression is true
    • May not run at all if expression is false when execution reaches the loop
    • Does not require a body if the controlling expression can handle all required functionality
  • For loops
    • General form: for(initialisation; condition; iteration)
    • Commas allow multiple initialisation and iteration statements
    • Initialisation and/or iteration statements may be absent
    • All three parts may be empty for an infinite loop
  • Enhanced for loops
    • General form: for(type variable : collection)
    • Iterates through a collection and places the current value in the variable
    • May be used for multi-dimensional arrays using nesting
  • Do/while loops
    • Always runs once, subsequent runs will occur if the controlling expression is true at the end of the first iteration.
    • Decrement/Increment may be included in the controlling expression as a pre-fix decrement/increment preceding the comparison
  • Break and continue
    • break terminates the loop and execution continues with the first statement after the loop
    • continue terminates the current loop iteration and execution continues with the next loop iteration
    • break label breaks out of the named block of code
    • continue label ends the current iteration of the block label and triggers the next iteration to begin
    • break may only be used within a switch block’s case or default labels or in a loop.
    • continue may only be used in a loop.

Methods and Encapsulation

  • Method arguments and return values
    • A method may accept values into variables which are known as parameters
    • A method that accepts parameters is passed arguments(variables or literals) when it is called
    • Void methods do not return values or include a return statement.
    • Non-void methods must return a value via the return statement.
    • The last parameter of a method may be a varargs parameter which may accept an arbitary number of values. The syntax is: type… array_name
  • Static keyword
    • Static variables are essentially global variables
    • Static methods can only directly call other static methods
    • Static methods can only directly access static data
    • Static methods cannot use this or super – it will cause a compilation error
    • Static members can be accessed before an instance of its class is created and without reference to an object.
    • Inside their class static members can be used without specifying the name of the class.
    • Outside their class static members can be used by specifying the class name followed by the dot operator and member name.
  • Overloaded methods
    • Overloaded method use the number and types of arguments to determine which version of the overloaded method to call.
    • Return type are not used to determine which overloaded method to call.
  • Default and user-defined constructors
    • If no user-defined constructors are provided for a class, the compiler will supply a default no argument constructor
  • Create and overload constructors
    • If a class will be inherited, a no argument constructor should be supplied.
  • Access modifiers
    • public
      • Can be accessed by any other code
    • private
      • Can only be accessed by other methods in its class
    • protected
      • Applies only with inheritance
    • default
      • Also known as package-private
      • Accessible from all classes within its package
  • Encapsulation
    • Declare variables of a class private
    • Provide public setter and getter methods
  • Call by reference vs call by value
    • Primitives types are passed by value – the method has a copy of the primitive and can’t change the original variable
    • Object types are passed by reference – the method can change the original object

Inheritance

  • Implementing inheritance
    • Java implements single inheritance
    • On instantiation of a subclass, the no-arg constructor of the superclass will be executed prior to the subclass constructor running
    • Subclass cannot access those members of a superclass that have been declared private
    • Declaring a method as final means it cannot be overriden by a subclass – a compile-time error will occur
    • Declaring a class as final means it cannot be inherited.
      • A class cannot be abstract and final
  • Polymorphism
    • Subclasses of a class can implement their own unique behaviour but share common functionality with the superclass
    • Achieved by overriding superclass methods to provide unique behaviour in the subclass, while also adding additional unique methods specific to the subclass
  • Reference type vs object type
    • A reference variable may be declared to be of a type that is a superclass of the actual object type
    • Reference type determines which members of the object are accessible
    • Only those members declared in the reference type will be accessible, so some members of actual object type may not be accessible
  • Casting
    • Where a superclass has been used as reference type, the object may be cast to the actual type to access members that are not part of the reference type
  • Super
    • May be used to call the constructor of a superclass – the signature will determine which constructor
    • May be used to refer to a method or instance variable in the superclass
  • This
    • this is a reference to the current object
    • this() can be used to invoke an overloaded constructor
    • this may be used to refer to a redefined instance member variable from a method.
  • Interfaces
    • An interface is a a reference type, like a class, that can only contain constants, method signatures, default methods, static methods and nested types.
    • Cannot be initialised
    • Are designed to be implemented by classes or extended by other interfaces
    • A class based on the interface uses the implements keyword to declare this.
  • Abstract classes and interfaces
    • Abstract classes cannot be initialised
    • Abstract classes are able to be subclassed, with subclasses inheriting the methods provide by the abstract class
    • An abstract class may implement an interface

Exceptions

Preparing for Oracle Java SE7 Programmer I (OCA) Certification Exam

Over the past couple of weeks, thanks to my work, I’ve been preparing to sit the Oracle Java SE7(OCA) Certification Exam. I already knew how to program in Java, having completed several units at University that covered Java programming therefore I was pretty confident that I would be able to pass the Associate level exam without any hitches. Just to be sure though, I asked to take the Oracle University 1 Day Exam Prep Seminar to brush up on anything I had missed. I was also given a practice exam license for the Kaplan Self-Testing for the Java SE7 Programmer I exam.

As it turns out it is pretty lucky I did get the Exam Prep Seminar and Practice Testing. On first attempt of the practice test I found it harder than I had expected.

Oracle University 1 Day Exam Prep Seminar

The Oracle University 1 Day Exam Prep Seminar is a fantastic way to prepare for anyone who has studied and/or programmed in Java in the past. While I had passed my Java programming units at Uni, those exams hadn’t required me to memorise anywhere near as much detail about Java syntax, development techniques and APIs.

The Exam Prep Seminar consists of 3 hours and 25 minutes of videos, divided into 8 sections covering the major syllabus areas required to pass. It’s presented as a slideshow presentation led by Oracle expert instructor Lee Klement. Each video is around 6-12 minutes long. The material is covered quickly but in a fair bit of depth. The detail is suitable for people who have a familiarity with Java. It certainly wouldn’t be suitable for a beginner.

The prep seminar does go through a few sample questions from the exam which is quite useful to indicate what the questions are likely to be like. The sample questions make it clear that a high attention to detail is required. The programs you’re asked to analyse in the exam are not necessarily written in the way real world programs would be. They are specifically designed to make sure you know exactly how each aspect of the Java programming language works. In some ways, many of them could be considered as “trick questions”.

I strongly recommend this seminar for anyone who wants to complete the certification

Kaplan Self-Test

The Kaplan self-test was useful to practice but hard to make work. It uses either a Windows 9x testing program you need to download or a Flash-based browser test interface. I couldn’t load the self testing program on my work computer due to administrator restrictions and I couldn’t load it on my personal Macbook. Most browsers now prevent Adobe Flash from being used for security reasons. I eventually managed to find an old version of Firefox I could use on my Macbook though. Making their self-test system work really felt like going back to the late 1990s.

Once accessed, the Kaplan self-tests offer a number of options. It offers a configurable learning mode, certification mode and flash card mode. It’ll let you either simulate doing a full certification exam or just practice the syllabus areas you need to. The questions are excellent at making you think. Once completed you can view answers including information that tells you why the correct answer was correct and why the wrong answers are not correct. This makes it quite useful to improve on the things you don’t know.

The report that comes out of it breaks down the areas you need to work on, and I’ve used those to focus my study. See my study notes for the key things I thought I needed to work on.

The Exam and My Results

I undertook my exam at the Dialog Pearson Vue testing centre in South Brisbane. The setup is pretty much the same as every other certification testing setup. You’ve got a computer running just the test software.

The exam was 120 minutes and had 70 questions. They were all multiple choice and for some you will need to select more than one answer. As expected from the practice test, a lot of the questions felt like trick questions. They often use keywords and statements in ways you would not ever expect to find in production code and you may therefore be unfamiliar. They are used to test your knowledge of syntax rules and API items.

I used all 120 minutes. I finished the last question with 8 minutes to go and used the remaining 8 minutes to review the questions I had issues with. You are able to mark ones you want to review. The review was useful and I think I changed a few important errors there.

I got a score of 73%. The pass mark was 63%, but this may vary as question sets vary. So I passed first time! I definitely think my extensive study, including writing up study notes after I had issues on the practice test, definitely helped me succeed.

One last note, you won’t get your result immediately. The test instructions say it will be emailed to you, but it wasn’t for me. I had to log into the CertView site to view it. I did it an hour after completing the test and my results were available. The certification apparently follows up to 48 hours later.