Wednesday 2 September 2015

NOTE: SAS "Inside" of Hadoop

We previously looked at SAS Grid Manager for Hadoop, which brings workload management, accelerated processing, and scheduling to a Hadoop environment. This was introduced with the m3 maintenance release of SAS v9.4. M3 also introduced support for using an Oozie scheduling server.

If you're keen to get additional SAS services running on your Hadoop cluster, potentially reducing "data miles", you'll be pleased to know that SAS has an experimental feature in v2.7 of the LASR Analytic Server that allows us to experimentally manage resources with YARN. I need to stress "experimental" - this is not ready for our production systems quite yet, unless reliability & availability are not our top priorities.

If the experimental status doesn't put you off then you can find more details at the back of the LASR Analytic Server 2.7: Reference Guide.

YARN (Yet Another Resource Negotiator) is part of the base framework of version 2 of Apache Hadoop. It's a resource manager and it takes care of the Hadoop cluster's compute resources. YARN can manage and share resources between various applications. Configuring LASR to participate in YARN's resource sharing allows YARN to have a complete picture of activities on the cluster.

The use of YARN with LASR is part of the increasing integration between SAS and Hadoop. I look forward to seeing it move to "general availability" status.

Tuesday 1 September 2015

NOTE: Visual Analytics V7.3 Has Arrived

Maintenance release 15w33 of SAS v9.4 arrived in August and it included a new version of Visual Analytics - version 7.3. This new release doesn't offer significant new features over its previous releases but I did notice that Visual Analytics Viewer's default appearance is now MODERN instead of CLASSIC. This is related to VA's transition from Flash to HTML5 technology - something I mentioned back in July.

I did also notice that VA v7.3 comes with an option to allow us to print the entire content of tables, crosstabs, gauges, and containers with content that is only partially available in the layout of the report section. I'm sure that'll be handy on occasions - especially for those of us with small screens!

Tuesday 25 August 2015

Making Notes With Livescribe 3

A couple of weeks ago I wrote about active listening, and I mentioned that I find that note-taking helps me understand (and remember) what I am being told. The downside of this approach is the pile of age-old notebooks that I don't want to throw away "just in case I need them". I think I recently found a solution to my problem...

Back in June I bought a Livescribe 3 digital pen. This is a conventional ink-based pen that writes notes in books, but its party trick is that it has a miniature infra-red camera built-in that observes my jottings and stores an electronic copy. The copy is stored inside the pen, but a Bluetooth link with my Android phone (or Apple iPhone) sees the digital images transferred to my phone (either in real-time, or next time I connect the pen and phone). The Livescribe+ app on my phone allows me to view the pages and books of all of my writings since I bought the pen in June. I can throw away the books I've used since June!

The Livescribe 3 is the latest in a series of smart pens from Livescribe. Like its predecessors, the Livescribe 3 needs the company's proprietary micro-dotted paper for it to do its magic, but the books are reasonably priced and widely available - or you can print your own micro-dotted paper if you wish (if your printer can manage 600dpi+). In addition, the pen uses Livescribe's proprietary ink refills (67mm length and 2.35 diameter).

Truth be told, the pen is a bit chunky, but I soon got used to its shape and I no longer notice it. This may, in no small part, be due to its light weight. All-in-all it feels very comfortable in my hand.

I've found the digitised images to be very accurate and readable facsimiles of my analogue scribbles. And the Livescribe+ app does a decent job of optical character recognition (OCR) of my words too - making the digital pages searchable.

If you're worried that your notes are a little vulnerable if they are only stored on your phone, fear not because the iOS version of the Livescribe+ app allows automatic synchronisation to Evernote and Microsoft OneNote. Apparently this feature will be added to the Android app soon.

All-in-all I'm smitten with my new pen. It accurately digitises and stores my scribble & writings, and it is nice and easy to use.

The Neo N2, which grew out of a Kickstarter campaign, is a direct competitor and gets good reviews. I honestly can't recall why I chose the Livescribe over the Neo. I think they're pretty similar in features and price.

Did I forget to mention the price of the Livescribe 3? Ah, this is its only major drawback. Expect to pay about £130.

Wednesday 12 August 2015

NOTE: SAS Global Forum 2016 - Call for Content is Open #SASGF

Have you thought about sharing some of your knowledge with fellow SAS practitioners at SAS Global Forum 2016 in Las Vegas, April 18 - 21? User participation is what makes SAS Global Forum (SGF). Attending the papers from SAS staff is always informative, but user contributed papers provide knowledge from real-word experiences and are equally valuable.

If you're interested, or unsure, take a look at the Call for Content section of the event web site. You don't have to be the world's most eloquent speaker, nor do you have to be a SAS guru. Forum attendees are always keen to pick up nuggets of information from fellow attendees.

The benefits to fellow attendees are obvious, but there are several benefits for speakers too. It's great for professional development; it will reflect well on you with your employers or clients; and you will be surprised how much you will learn as you go through the writing process. And if you're still not convinced, be assured that you can supported by a mentor too.

Go on, give it a try!...

Note how the event appears to have shifted one day later in the week than normal, i.e. in 2016 it will start on a Monday (pre-conference tutorials?) and then run from Tuesday to Thursday. And the favoured twitter hashtag no longer contains the year. Goodness, it's all change at SGF!!

Tuesday 11 August 2015

Good Communicators are Good Listeners

A couple of weeks back I wrote how great developers have skills beyond syntax and design patterns. One of those key skills is communication.

Communication is a two-way operation, and the art of listening is oft overlooked, so I thought I'd offer some notes on the subject here. Specifically, what is commonly known as "active listening".

There is a difference between simply hearing someone’s words and engaging & understanding what someone is saying. If we want to be an active listener, it isn't enough to just hear what someone is saying. Active listening means i) we are focused on what is being said, and ii) we are open to the speaker's point of view.

For the first part, paying attention, we should be listening and trying to understand what this means. There are different means of doing this, which suit different people. For me, I do this better by taking notes (plus, writing down what people say makes them feel heard and important!) and by using follow-up statements like:

  • Can I try to repeat back what you said, in my own words?...
  • So, another way of saying that is ..... Is that correct?

Each of these makes sure we understood the person's point of view and gives them confidence that we are doing so.

The second part of active listening is being receptive to other peoples' thoughts and ideas. Sometimes this can be really difficult for developers, or other smart people, because many times they are 5 steps ahead of everyone else! However, interrupting or assuming what someone else is going to say can hinder us from hearing new ideas.

It is very important to make people feel heard (not just listening to their words, but understanding their points).  To be more receptive to others here are some tips:

Don't interrupt.  This one can be hard, when we already know the answer to something, or have thought through the solution or path already. However, allowing people to say their thoughts shows respect and consideration for others.

Don't assume what they are going to say. We might be right, but we could also be wrong, and either way it prevents the other person from voicing their ideas. Problem solving, brainstorming, and planning are all things that can be better done collaboratively, and that means each person should have a chance to contribute.

Do ask clarifying questions. Try to understand the "why" behind what they are suggesting; even if the "how" is wrong, their motivations may be sound. Getting to the root of things by asking questions makes the other people feel heard out, ensures the roots of issues get addressed, and can lead to more productive conversations.

Acknowledge their view as important. They probably wouldn't have said anything if it didn't mean something to them. Therefore acknowledging their point, or motivation behind their point, makes the other person feel recognised and their concerns addressed. A simple statement like "that is a good thing to consider" or "thanks for bringing that up, we should definitely note it" is all that is needed; even if you plan to refute that view or disagree with it.

Wait for their response. If we ask someone a question show that we want to hear the answer by waiting for him or her to respond. Some people (especially technical folks) tend to like to think things through before they verbalise them. Other people are more comfortable when they are talking. If you fall into the latter camp, make sure that if you are talking to someone who likes to think through their reply, that you don’t keep talking and wait for their answer. Ask the question, pause and wait.

Notice their body language. Not all communication is said out loud, so take a few moments and try to observe how the person is reacting to what is transpiring. Are they guarded? Visibly upset or shaken? Are they excited and happy, nodding along to the statements? These are all clues that can help us understand what is really going on under the covers. Learning to pay attention to these can help us change our tone, or adjust our approach.

Listening (and contemplation) is such an important element of communication. We should always remember that we have two ears yet only one mouth; we should use them in proportion!

Tuesday 4 August 2015

Unix Command-Line Top 10

If you are introduced to command-line Unix (or Linux or any other Posix-compliant variant) after a lifetime of Windows, it can be a daunting experience. To those thrust into this position, I always offer my top "ten" most useful commands to begin getting to terms (and getting on top of) the new environment.

  1. "pwd" and "cd" - when sat at the command-line, you must imagine that you are sat "in" a directory ("folder" in Windows-speak). All of your commands will be aimed at the directory in which you are currently sitting. To aim a command at a different directory you must either "move" to that directory, or specify the other directory's name in the appropriate position within the parameters of your chosen command
    pwd - present working directory - tells you what directory you are currently sat in
    cd - change directory - move to another directory
  2. "ls" and "ls -l" - to see what files and subdirectories live within a directory, use either "ls" to see a simple list or use "ls -l" to get a detailed list that includes modify date, owner, and permissions
  3. "more" - if you want to see the contents of a file, more will do that for you. When you "more a file" you will see the first page of it and will then have the option of issuing further keyboard instructions:
    <space> - scroll down one page
    <return> - scroll down one line
    "q" - exit from more back to the Unix command line
  4. "grep" and "find" - use "grep" to search the content of files; use "find" to locate files with specific patterns of names
  5. "groups" - the "groups" command will tell you which security groups you are a member of
  6. "chmod" and "chown" and "chgrp" - these three commands respectively change the security permissions on a file, the individual owner of a file, and the group that owns a file (yes, in unix each file has two owners)
  7. "vi" - the much feared but ultimately powerful file editor
  8. "ps" and "top" - show what processes are running, and show the highest users of system resources
  9. ">" and "|" - Not strictly commands, but a most valuable adjunct to the use of commands, and one of the key features of Unix. The principle of Unix commands is to do one single task and to do it well. The principle of using Unix commands is to join the together by "piping" the output from one command to the next [to do]
  10. "man" - and if my brief explanations are not enough, view the pages of the manual with the "man" command
The foregoing is a vastly simplified description of the "ten" commands. Use the man command to find more information on each of them. The web is full of similar pages and documents with commands and how to use them, so there are plenty of sources for further information.

Are you an experienced Unix command-line warrior? What are your ten most frequently used commands?