Archive for the ‘IT’ Category

Keep Linux Permissions and Groups for ITAR work in synch Automatically with Excel

Monday, September 10th, 2012

Engineers using Linux clusters often struggle with keeping permissions for various projects in sync, especially for ITAR or other sensitive projects. Analysts are often brought on a project, moved around, and group and file permissions can get out of sync quickly with who should actually have access.

TotalCAE recently implemented a mechansim where a CAE manager can list in an Excel spreadsheet a list of projects, and select via an Excel drop down menu a list of analysts that are to work on that project. From this spreadsheet, custom Linux software reads the spreadsheet from the file share and generates and syncs all required groups, permissions, and ownership to make this process simple and maintainable. For new projects, a default directory structure is created. A full audit trail is maintained, and users can be removed, added, or migrated to different projects without having to keep the Linux group and file permissions in sync. The code was written in Python using xlrd.

An example workflow is below:

This project helped the CAE engineers and managers focus on their CAE work, and less on being Linux gurus.

Solver Fails with Input Deck Edited on Windows, and Ran on Linux.

Friday, March 2nd, 2012

Have you ever experienced a Linux solver failure with Fluent etc. when you edited the input deck or include files on a Windows workstation? If so, this post unravels the mystery of why that happens, and how to prevent it.

The Fangled Newline

In text documents lines are separated by what is called a newline, also called an end of line (EOL). Different operating systems have different invisible codes that represent this newline.

For example, on Windows the end of line is marked by a carriage return (CR) followed by a line feed (LF). On Linux/Unix, the end of line is marked only by a single line feed (LF)

This means that text documents that come from a Windows system won’t always work on Linux with your solver. Some utilities won’t show these invisible end of line characters, but sometimes you will see the CR on Linux as a bunch of ^M if you use an editor like vi.

The Problem

Here is an example of a journal file created on Windows, and is having problems with a Linux solver. I use the “cat” command on Linux, to view the file, and note it looks completely normal:

[~]$ cat test.jou
file
read-case-data/test.cas
quit

However, by adding the “-v” option I can see the special Windows end of line Carriage Return (CR) represented by ^M, this indicates this file is a WINDOWS formatted file, and will likely cause the solver to fail with strange errors:

[~]$ cat -v test.jou
file^M
read-case-data/test.cas^M
quit^M

The Fix

A simple way to fix this is issue the “dos2unix” command on Linux

[~]$ dos2unix test.jou
dos2unix: converting file test.jou to UNIX format ...

Now a quick check shows no ^M:

[~]$ cat -v test.jou
file
read-case-data/test.cas
quit

Preventing It

Instead of running dos2unix on Linux every time you edited it originally on Windows, I recommend to use the free notepad++ editor on Windows that lets you control end of line handling. In Notepad++ go to Edit->EOL Conversions->UNIX Format to edit a file and keep it in UNIX (Linux) format.

The other major way end of lines get messed up is transferring files using FTP or other transfer mechanism that changes the line feeds automatically. If you use ASCII mode when transferring with FTP, this can happen. If you always use BINARY mode by typing the word “binary” at the FTP prompt before you use GET or PUT. FTP will not try and modify the end of lines if you force BINARY mode, even though the file is ASCII, preserving the source encoding.

We will be doing a series of posts on Linux/Windows interoperability. Stay tuned!