NOTE: DI Studio Version 4.3

Version 4.3 of SAS Data Integration Studio (DI Studio) is to be released alongside SAS 9.3 later this year. I stopped by the Data Integration booth at SAS Global Forum to get some detail from those that should know. There are a raft of enhancements that are worthy of note, but the big one that caught my eye was job versioning and rollback. Here's what I found (and saw)...

• Increased ability to perform ELT (Extract, Load and Transform) in addition to traditional ETL - doing your transformations down in the database and thus minimising the data miles travelled by your data
• Code import wizard in 4.2 already understands macros to the extent that it can create and display a process node representing the macro, but DI Studio 4.3 will give you the option of expanding the macro so that you see the component tasks of the macro in the your DI process flow. This gives you a clearer picture of what your job is doing and allows job performance to be collected at a more granular level
• Talking of job performance, DI Studio 4.3 provides more breadth and depth with regard to the performance profile of your jobs and their steps. You have access to predefined reports that can (optionally) be created in Web Report Studio
• A Type 1 SCD loaded transform has been added for those that don't need the complexity of the Type 2 loader. Not sure what Type 1 and Type 2 are, see my earlier article on the subject
• Job deployment can now be done from an OS command line using a newly-supplied shell script. This script supports deployment of one or more jobs at a time, and it can be scheduled. For those of you who have invested in automating the promotion and deployment of your jobs using the programmatic interfaces to export/import that DI Studio 4.2 brought, you can now consider completing the task by automating the deployment of your promoted jobs. Apparently, job deployment hasn't been made available through the programmable API because the task of deployment requires a great deal of Java activity that SAS weren't comfortable with delivering through the APIs. Disappointing, but there you go
• And finally, for those who take configuration management seriously, we have versioning of jobs (and packages). Sadly this isn't a top-down delivery of a release management capability, but it's a step in the right direction. SAS have provided plug-ins for CVS and Subversion, but they say they'll publish the API and allow you to write your own plug-ins for your own preferred source code management system, such as IBM Rational's ClearCase. These plug-ins facilitate the process of pushing exported packages (with one or more jobs and dependent objects) into your source code management system. From within the DI Studio interface you can see and inspect versions of your packages.You can even compare objects within packages, so you can answer the common Release Manager questions like "what changes did you make to the job between version 2 and version 4?". Sadly, the comparison can only be done on one object at a time.

So, all-in-all, DI Studio 4.3 has a lot to offer alongside the introduction of SAS 9.3. The versioning leaves a lot more work still to be done with regard to release management, support for a wider range of source code management systems, and detailed reporting. Nonetheless, it's a job well done by the DI Studio team.

The plug-ins are not available from Management Console and thus cannot be used and applied to any SAS metadata objects that you might expect to export/import. Thus there's no support for BI objects. That's disappointing, but SAS tell me they are aware of the shortcoming and hope to plug the gap in a future release.

[See all of my posts regarding SAS Global Forum]

NOTE: SAS Global Forum 2011 - Update #2

Day 2 at SAS Global Forum (or is it day 3? I'm not sure) and my feet are aching due to the enormous amount of walking required. Yesterday I spent a lot of time in the Demo Area, talking to SAS developers about their work in the areas of DI, Mobile BI and In-Memory Data Viz. I'll put together some posts about those things later today.

Meanwhile, it's grid, grid, grid all the way today. I'll be camped in the Systems Architecture stream for most of the morning. I'm currently working on a large grid project, so this stream is perfect for me. ("large grid project", is that a tautology? Aren't all grid projects large?)

I can't stay in the Systems Architecture stream all day though because I'm presenting my V-Model paper in th Coders' Corner stream at 5:40pm today. I'm looking forward to it. If you're at SAS Global Forum, be sure to come along and say "hi".

[See all of my posts regarding SAS Global Forum]

NOTE: Get a Clearer View of Your Security Metadata

The partners booths at SAS Global Forum rarely offer third-party SAS software products. The one perrenial exception is Futrix (offering self-service BI beyond SAS's own capabilities), but this year sees an offering from Metacoda. It's the Metacoda Security Plug-Ins. This product is worth a serious look if you take the security of your SAS system and its data seriously.

SAS's security capabilities are not in question. They are multifarious, comprehensive and flexible. But that scope of capability means they have many dimensions, and your site's security implementation can quickly become difficult to perceive. That's where the Metacoda Security Plug-Ins step in to help. They are offered as a plug-in for SAS Management Console, and they offer a large number of views on your security metadata, meaning there's sure to be a convenient view to match your current interest and need.

The current version of the plug-ins work with 9.1.3 and 9.2. It covers ACTs, ACEs, users, groups and logins. A new version of the plug-ins will be available by summer and will add roles, capabilities and further details on internal logins for 9.2 customers.

I was impressed with what I saw demonstrated on the booth. Combined with flexible and affordable licensing models, I recommend you take a look and contact Metacoda for further details.

[See all of my posts regarding SAS Global Forum]

NOTE: SAS Global Forum 2011 - Update #1

There's a lot for the 3,000+ attendees of SAS Global Forum 2011 here in Las Vegas to see and do. The opening session and this morning's Technology Connection session offered plenty of snippets of information. I'm now attending papers and cruising the demo area in order to get more detail on what I already heard. The conference centre is huge, so the cruising is pretty tiring!

Here are some snippets of what I heard so far:

• The SASware ballot resulted in commitment from SAS to provide the ability to run stored processes from Base SAS code and to provide vector graphics from ODS (thereby permitting resizing and scaling of the output images). No release date was offered for these features
• SAS are really pumped about their announcements made yesterday morning regarding high-performance in-database analytics for Teradata and Greenplum. Examples of turnaround time being reduced from 20 hours to 80 seconds were offered, based upon massively parallel processing. Billions of rows, with thousands of columns. Use of smart techniques instead of brute force, i.e. minimising distance travelled by your data and maximising communication between processes
• Good, old-fashioned display manager is still receiving updates. ODS will be switched on by default in 9.3, so you don't need to code so many ODS statements; graphics output will be displayed in the in-built browser rather than the old graphics display tool, hence providing greater flexibility and proper response to drill-downs
• ODS Graphics will be part of Base SAS in 9.3
• Enterprise Guide 5.1 offers type-ahead for column names in addition to the existing ability for library names and table names
• SAS Conversation Centre (available May 2011) will offer the ability to manage and take control of Twitter interactions based upon sentiment analysis

Videos of the key sessions are all available on the SAS web site. One of the highlights for me was listening to Gary Loveman, chairman of Caesars Entertainment. Gary spoke most eloquently and entertainingly about his company's success in using customer intelligence.

[See all of my posts regarding SAS Global Forum]

NOTE: Mobile BI

I dropped and broke my beloved HTC HD2 Windows Mobile smartphone just before Christmas. I loved its large 4.3" screen and its neat HTC Sense interface over the top of clunky Windows Mobile V6.5. I've replaced it with an LG Optimus One (P500). Why should this interest the SAS world? I'll tell you...

My sad telephonic experience has given me the opportunity to get some insight into a topic that I'm expecting to be hot at this year's SAS Global Forum. I'm talking about Mobile Business Intelligence. It sounds geeky but I'm convinced there's real business value and it'll be a hot topic in 2011.

I enjoyed the large screen on the HTC. I found it big enough to use the on-screen keyboard and to type emails without great inconvenience. It allowed me to view web sites perfectly adequately too. But it lacked support from developers and I was constantly frustrated by the launch of new apps for Apple's iOS and Google's Android, but not for Microsoft's Windows Mobile. So, when it broke I thought I'd take the plunge into the world of Android. Hence my purchase of the LG Optimus One.

I'm very pleased with the LG, and I love the availability and variety of apps (including TripIt to help me plan my trip to SAS Global Forum), but I do find the smaller 3.2" screen to be a frustration. I no longer find it so comfortable to write emails and browse the web.

As a newbie to the Android world, I've been keen to follow the developments of the Android tablets that have appeared over the last few months (first the Samsung, then the Motorola, and now everybody's jumping on the bandwagon). Given my recent experiences and frustrations, I can see how an Android tablet could be just what I need to keep in touch while I'm travelling.

I've never been a fan of Apple's expensive and locked-in approach to their products (though I do fully appreciate the gorgeous aesthetics of them). Hence I haven't given the iPad much attention. But I now see how the iPad and its cheaper, more-open Android cousins offer the mobile business person a fantastic tool for doing their job. Whether it's keeping in touch with colleagues and customers, having instant access to operational details like stock availability and product pricing, or being able to view strategic company information when you want and where you want, these tablets offer a convenience that conventional laptops and netbooks fail to deliver.

A lot of business intelligence (BI) vendors have released mobile products in order to meet the demand, firstly for the iPad but more recently for Android (and doubtless BlackBerry's upcoming PlayBook tablet will soon be supported too). BI vendors must provide their customers with the ability to design once and render anywhere; to support BI anywhere (desktop, browser, tablet, smartphone) and in any interface (dashboard, visual exploration, query, or Office). Leading industry analyst Howard Dresner recently published a study on the mobile BI market. Information Builders came out among the top contenders and chose to publish the study in full. Dresner concluded that "within two years a significant percentage of users will consume BI content exclusively on mobile devices".

SAS's recently signed partnership with MeLLmo Inc, creator of Roambi, one of the leading business apps for interactive mobile analytics, gives it the ability to deliver mobile SAS BI - albeit only for Apple iOS products at the moment. As I said earlier in this article, I expect to hear a lot about this partnership, and to see plenty of demos of what it can deliver, at SAS Global Forum. I'll be keen to not only look at the interface but to also understand the infrastructure that is required to allow people to work and move around transparently between office, mobile, online and offline modes.

However, I'll make one final point. Mobile BI is big and getting bigger, but there are plenty of other key developments in the industry that SAS must follow to retain their eminent position. Cloud-based delivery of services, and in-memory analytics are just two that I also hope to hear more about at SAS Global Forum.

Time to pack and find my passport...

NOTE: Positive Moves in the SAS Blogosphere

I'm pleased to see today's announcement from SAS regarding the merge of two SAS blogs (The DO Loop (written by Rick Wicklin) and The SAS Dummy (previously written by Chris Hemedinger)). As the announcement makes clear, change was needed. The new blog, written by Wicklin, will appear next week. You are encouraged to read the announcement for further detail.

There's related information on Adam Cooper's F1 blog.

Posted by: Andy Ratcliffe
Date: April 1, 2011