How to sort 1TB of data in 68 seconds or even 1 PB

“We are excited to announce we were able to sort 1TB (stored on the Google File System as 10 billion 100-byte records in uncompressed text files) on 1,000 computers in 68 seconds.” – Google Blog

But that was not enough they tried with 1000 times more data. 1PB.

“It took six hours and two minutes to sort 1PB (10 trillion 100-byte records) on 4,000 computers.”

The method they use is MapReduce that makes it possible to run multiple processes simultaneously on multiple computers.

Programming model for MapReduce

Input & Output: each a set of key/value pairs

Programmer specifies two functions:

map (in_key, in_value) -> list(out_key, intermediate_value)

• Processes input key/value pair
• Produces set of intermediate pairs

reduce (out_key, list(intermediate_value)) -> list(out_value)

• Combines all intermediate values for a particular key
• Produces a set of merged output values (usually just one)

Inspired by similar primitives in LISP and other languages.

Adobe Updater problem solved

AdobeUpdater.exe started to show an annoying empty dialog box each time I restarted the computer. It said the installation was not ready and I could continue it later. I found no way to clearing it or accessing the settings or stopping it from showing up again and again.

I tried installing the latest Adobe Updater 5.1.1 update to no avail.

Eventually I solved it by deleting the Updater5 folder (I have Windows Vista):

%USERPROFILE%\AppData\Local\Adobe\Updater5

HP+Autodesk+NVIDIA deal

If you’re about to purchase both AutoCAD 2009 and a new PC this is worth taking a look at.

Each of the bundles include:
•AutoCAD^® 2009 Software, the latest version of the world's leading 2D and 3D CAD application.
•HP xw4600 Workstation with Intel^® Core^™ 2 Duo E7200 processor offering next-generation performance technologies in a powerful, flexible and reliable workstation.
•NVIDIA Quadro^® Professional Graphics providing unmatched power and performance.

Visit www.hp.com/go/cad and NVIDIA’s Autodesk site www.nvidia.com/builtforpros.

BTW. I noticed in NVIDIA’s newsletter NVIDIA Tesla Personal Supercomputer. It is based on the NVIDIA CUDA parallel computing architecture and powered by up to 960 parallel processing cores. I wouldn’t mind having that as a rendering machine.

3D Printing Service for AutoCAD models

This Autodesk website has links to several services:
Print an AutoCAD Model Through a 3D Printing Service

Here is a document describing how to optimize your models and prepare them for 3D printing. You can print 3D solids, but not surfaces, parts of a block, meshes, 2D
geometry or text.

This is related to the AutoCAD 2009 subscription bonus pack 3 Shaan has posted about.

Small Basic - for kids or if you want to learn programming

I started programming on Sinclair ZX81 with a built in Sinclair BASIC interpreter in the early 80:ies and think it is great to see a brand new, simple and free programming language and IDE.

Visit Small Basic to download it and for more information.

Small Basic starts with a really simple programming language that gathers inspiration from the original BASIC language.  It has no more than 15 keywords and is strictly imperative.  There are no classes, scopes, generics, lambdas, etc. - just pure imperative code.  The language is typeless and all variables are dynamic and global all the time.  The code gets compiled to IL and runs on the .Net Framework.

It comes with a set of libraries that can be accessed from within a Small Basic program.  Since the language itself is .Net based, new libraries can be created or the existing libraries modified using any .Net programming language.

Next, it combines the features of the language and the libraries into a very simple and friendly programming environment.  This environment gives beginners, access to professional features like Intellisense(TM) and Instant context sensitive help.  It makes writing programs and executing them a breeze.

Small Basic blog

Project Newport real-time 3D story building technology

Project Newport involves a mix of architecture, movies, and games. This sounds really interesting.

Convey design intent with Project Newport – real-time 3D story building technology for architectural visualization and presentation. With breakthrough game-engine technology, Project Newport is easy to use and enables architects to show their designs in context.  They can explore design options, visualize changes instantly, and create vivid, immersive 3D presentations.

Keep an eye on Project Newport

Via the Autodesk Lab

Wonder how it might relate to or have similarities with Wikitecture

ADN member access to Autodesk Subscription site

Jim Quanci just emailed me that “You have access to the Subscription Center including all extensions now - see right hand sash of the ADN extranet home page - http://adn.autodesk.com”

As an ADN member, you have “Visitor” access to the Autodesk Subscription Center for two uses:

1. Access to quarterly extension releases for add-on design and testing purposes. As with all Autodesk software provided to ADN members, these quarterly extension releases may not be used for end-user “production” design and modeling purposes – and may not be shared with end-users or outside your company.
2. Access to on-line training classes so you can quickly learn and understand how Autodesk products are designed to be used by end-users – enabling you to better design and integrate your technologies with Autodesk products.

This is really great news for Autodesk Developer Network members. Now I can also read “Stoeckel on Subs”. I find no feed though.

Process Explorer v11.3 now with physical memory history graph

Process Explorer v11.3: This update to Process Explorer includes numerous enhancements and bug fixes, including a physical memory history graph, options to configure memory tray icons, asyncronous thread symbol resolution and security ID lookup, dynamic recognition of new volume drive letters, multiple character matching in the process view, and a smaller memory footprint.

Notice the new tray icon.

Read also the excellent post Pushing the Limits of Windows: Virtual Memory on Mark Russinovich's technical blog for interesting facts about physical memory, address space, virtual memory, committed memory, paging file and more. Learn more about what’s in the task manager or even better the Process Explorer.

Via Sysinternals Site Discussion

Microsoft anti-malware protection for free for consumers

If you are about to planning for purchasing antivirus software consider this.

In hopes of getting more consumers protected, this week, Microsoft announced that it will offer a new security solution to consumers at no cost beginning in the second half of calendar year 2009. This new offering will focus on getting the majority of consumers the essential protection they need by providing comprehensive, real-time anti-malware protection, covering such threats as viruses, spyware, rootkits, trojans, and other emerging threats, in a single, focused solution.

In order for us to focus on delivering this new security solution to millions of customers around the world, we have decided to phase out Windows Live OneCare and will discontinue retail sales of Windows Live OneCare subscription service beginning June 30, 2009.

Question is why? OneCare even have outsold McAffee Virusscan. It probably is another goodwill thing to make Microsoft and Windows look better. Or maybe there was no future in OneCare as it was bloated with printer sharing, networking and performance tuning. Another question is how this will affect other third-party consumer security products.

Via OneCare blog

Update

We know that there are still some 60 percent of consumers in developed markets, and even more in emerging markets, that don’t have up-to-date security protection on their PCs, and we want to help provide that core level of protection.

…core malware protection in low-bandwidth scenarios, or that will run on less powerful machines. That's why we will design “Morro” to have a smaller footprint and to use fewer computing resources.

Via PressPass Q&A: Amy Barzdukas, senior director of product management, discusses Microsoft’s strategy to provide broader access to critical anti-malware protection.

Acrobat crash on startup - solution

I got a problem with Adobe Acrobat 8.1.3 Professional on Windows Vista SP1.

I opened a document and after a few seconds looking at it, it disappeared.

I tried a repair and then it got even worse. It was not even related to that document.

Only a few seconds after I start Acrobat it crashes with no error message whatsoever. Acrobat just quits and exits silently.

I tried to run Acrobat as Administrator and then it worked just fine.

Then I though about using the general trick to delete everything in the TEMP folder. That solved my problem and hopefully yours if you run into this.

This also solved a problem with the Adobe PDF writer that I started to have recently.

Free Autodesk certification voucher at AU

Great news for those attending Autodesk University. I wish you a great time at AU.

Via Beside the Cursor

Related: Autodesk Certification Program in Northern Europe

Move all file types with Live Mesh even Outlook PST files

There are some limitations with Live Mesh even though it is a promising product. Some files and folders cannot be shared or synchronized. According to this help document there are limits that I sometimes need to work around as I want to use Mesh to move data from one PC to another.

I need to sync even files with these file extensions:

• .bak
• .lnk (links / shortcuts)
• .pst (Outlook personal folders)
• .tmp

My solution was to zip the folders I needed to move/copy and then I synchronized the zip files. For the zipping I use the excellent freeware 7-zip.

I wanted this to happen peer-to-peer only so for this purpose I stopped synchronizing with the Live Desktop for the folder I used.

The Live Mesh forum is found here.

Purge layers that don’t want to be purged in AutoCAD

Want to clean up you drawings from extra or garbage layers?

The obvious way to purge layers is to use the Purge command but sometimes it’s not enough. Purge might need to be run several times. Make sure to select purge nested items.

I got a request per email and the not so obvious tip that worked was the following:

Try to open the drawing with RECOVER or run AUDIT on it and then run PURGE and select all and also purge nested items and it will probably go away. Sometimes this needs to be repeated several times.

Notice that you cannot purge

• the active or current layer
• layers with objects
• layer 0
• layer Defpoints (It can be purged if you remove all dimensions)
• layers in XREFs (xref-dependent layers)

In the Purge dialog box you can find items you cannot purge.

Layers can exist nested in blocks. They can be used by empty text strings (QTEXTMODE can be used to show these) or lines without length or even viewports that have the layer frozen.

LAYMRG (Layer Merge) or LAYDEL (Layer Delete) might be able to help you either merge the layer to for example layer 0 or if you don’t care what’s on the layer delete it.
LAYWALK is also useful to see what is in each layer and can purge unreferenced layers.
LAYTRANS can also purge layers. In the Layer Translator, right-click in the Translate From list. Click Purge Layers.

To use the filters in the Properties palette (QSelect) can also work but you might need to explode everything several things first.

Another way to generally purge a drawing is to Wblock the entire drawing to a new drawing or use –WBLOCK and * for whole drawing or you can Wblock selected objects.

You can also try to isolate the problem by deleting 50% of the drawing, try to purge and then keep repeating it until nothing is left. If the layer is still there it might be used by some kind of object that is not purged using the plain purge command. It might be something in a vertical like AutoCAD Architecture and then you have to maybe purge using the style manager.

DXF can also help. Save as DXF or use DXFOUT command and open in a text editor and search for the layer to see where it is used.

SmartPurger can help you automate your purging and scripting work.

What is your trick to purge layers?

AutoCAD Architecture on Wikipedia

I noticed that the AutoCAD Architecture page on Wikipedia was really limited so I have edited it. Please head over to the page and add to it or make further changes. If there is anything that is unclear you can add to the discussion page to discuss about the content.

I think I got the release dates correct for ADT v1 up to ACA 2009.

Maybe add something on BIM. Personally I’m among those that think that AutoCAD Architecture is more than only AutoCAD for architects.