Grip updated and ported to Windows

Recently I was working on my indexed grep. Now it could be used with Boost library, as alternative to POSIX. There are build for Windows in the release page. This version could have issues when grepping file with lines longer than 16 kB due to my simplified implementation of getline. I will probably replace it with Boost code, eventually.

There is also significant performance improvement of index generation (gripgen). Now reading file names is not blocking indexer. Also memory organisation was changed to play nicer with cache. As a result indexing over SSD drive is several times faster. Unfortunately, mechanical drive is usually the bottleneck itself, so it will be not much difference. Lower CPU load maybe. And under Windows indexing is painfully slow due to Windows Defender (at least during my test on Windows 10). It looks like Defender is scanning every file that I am opening for indexing. Excluding binary files might help, I guess.

And since file list is read asynchronously, now I could print pretty percentage progress. Of course gripgen can do it after it reads entire list, thus feeding it with find will not show you percentages until find ends.

I was also cleaning up the code, refactoring build system and adding unit tests. Further changes will be mainly polishing and bug fixing, I think. Or not, if I’ll have some great feature idea. We will see.

Fixing Audiotrak Maya U5 firmware

TL/DR: I’ve fixed Maya’s firmware to work better under Linux: Audiotrak-Maya-U5-firmware-mod.zip

Part 1: whats broken

Maya U5 is an USB sound card with 5.1 channels support. It has pretty decent sound quality to price ratio. I bought one to use it with my Windows computer. And under Windows it works great.

Then, one day I tried to use it under Linux. As you probably guessed – it didn’t went well. System recognized this device as stereo card, sound went down randomly, volume control was not always working. Audiotrak supports only Windows and OSX, but since this is standard USB audio device it should run under everything.

So of course I asked Google for help, and found firmware that should work. It was better – system detects proper 5.1 device. But other issues still occur.

My second step was to check official website for newer firmware updates. Well, there is one. Great! Well, actually not that great. My system tells me again that I have only two audio channels. Going back to previous firmware. Except firmware flasher attached with older firmware won’t flash my updated card.

Ok, lets compare this two firmware archives. It looks like only four files is changed: flasher exe and dll, VIAFwUpd.ini and one of the bin files. I bet it is the firmware itself. But lets check the ini first.

[DFU]
Firmware=VT1728_20121101_v0.61_0102_05_Release10_Gyrocom_withQsound.BIN

Yep, our firmware blob. And yes, after I’ve changed it to point to the older firmware bin, I can flash it with the new loader. Great work, we are where we started.

Ok, go back to Linux, lets check what kernel knows about this device

$ lsusb -vd 040d:3401

Bus 001 Device 003: ID 040d:3401 VIA Technologies, Inc. 
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x040d VIA Technologies, Inc.
  idProduct          0x3401 
  bcdDevice            0.61
  iManufacturer           1 
  iProduct                2 
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength          574
    bNumInterfaces          4
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xa0
      (Bus Powered)
      Remote Wakeup
    MaxPower              500mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      1 Control Device
      bInterfaceProtocol      0 
      iInterface              0 
      AudioControl Interface Descriptor:
        bLength                10
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdADC               1.00
        wTotalLength          205
        bInCollection           2
        baInterfaceNr( 0)       1
        baInterfaceNr( 1)       2
		
(...)
		
      AudioControl Interface Descriptor:
        bLength                14
        bDescriptorType        36
        bDescriptorSubtype      4 (MIXER_UNIT)
      Warning: Descriptor too short
        bUnitID                10
        bNrInPins               3
        baSourceID( 0)          1
        baSourceID( 1)          7
        baSourceID( 2)         19
        bNrChannels             6
        wChannelConfig     0x003f
          Left Front (L)
          Right Front (R)
          Center Front (C)
          Low Freqency Enhancement (LFE)
          Left Surround (LS)
          Right Surround (RS)
        iChannelNames           0 
        bmControls         0x00
        bmControls         0x00
        bmControls         0x0a
        iMixer                 36 
		
(...)

There are errors in the USB descriptors. Terrific.
Ok, I found something called Thesycon USB Descriptor Dumper – very handy tool. And it told me interesting things.

Information for device MAYA U5 (VID=0x040D PID=0x3401):

*** ERROR: Descriptor has errors! ***

(...)

Endpoint Descriptor (Audio/MIDI):
------------------------------
0x07	bLength
0x05	bDescriptorType
*** ERROR: Invalid descriptor length 0x07
Hex dump: 
0x07 0x05 0x01 0x09 0x40 0x02 0x01 

(...)
Endpoint Descriptor (Audio/MIDI):
------------------------------
0x07	bLength
0x05	bDescriptorType
*** ERROR: Invalid descriptor length 0x07
Hex dump: 
0x07 0x05 0x01 0x09 0x60 0x03 0x01 

Yeah, more errors (this error is repeated 7 times for every Audio/MIDI endpoint).

Since I have never done anything more complex with USB descriptors than looking at lsusb output, at this point I had to read a lot. There is official USB documentation here, there are also more approachable descriptions like this and that. Anyway, after some heavy reading I was able to confirm that some descriptors were broken. Probably the mixer descriptor was the problem. There is some kind of array that should map inputs to outputs, but it is truncated. Truncated MIDI descriptors also could make some trouble, but since I rarely play any MIDI, it is not priority for me.

Ok, lets look inside the device. There is VIA Vinyl™ VT2021 codec and VIA VT1728A CPU. It is hard to find anything about this chip more that it is 8032 MCU. Which is something like 8051 with extra peripherals. But wait, there is something similar. VT1620A looks like older brother of this chip. Maybe it is not full spec PDF, but it is some starting point. And IDA handle 8032 assembly.

Part 2: lets fix it

USB configuration descriptor followed by other descriptors could be found at offset 0x15ec (in firmware image release F). My first attempt was to erase MIDI endpoints. Unfortunately, this bricked the box. Flasher stopped to recognise the device.

After second examination of the board, I found 8-pin IC labelled as 25VF512A. An 512 kbit SPI flash. But how to program it without proper hardware? Well, isn’t my Raspberry PI has SPI interface? After quick googling, I’ve found Flashrom. I’ve soldered 5 wires, connected them with Raspberry and was able to resurrect Maya. Back to square one.

Having working solution to unbrick the device, we could make some more intrusive (and complicated) modifications. Ideally it would be to fix every broken descriptor. To do that, we need more room than we have (some descriptors are truncated). So we need to relocate them. But where?

I dumped whole content of flash with my Raspberry. It has 64 kB, but only about 30 kB is used by the image. Rest of the space is empty. So at the end of data, there is plenty of room.

Next we need to find references to this data. 0x15ec could be found only at offset 0x1902. It is not code, and at this moment I couldn’t figure out how it is used, but since we have working backup plan, we are safe. I’ve copied descriptors at the end of image (offset 0x78a0) updated value at 0x1902 to point to the new location and zeroed descriptors data at original location. After flashing (with Audiotrak flasher) it worked!

After many tries I was able to fix mixer and MIDI endpoints. I’ve removed remote-wakeup attribute from configuration descriptor to prevent system from suspending it. Now it is usable under Linux, but still there are minor glitches. I’ve tried to fix also newest image, but could’t make 5.1 to work, so I’ve stayed with older one.

Here is modified (fixed) firmware with new loader, so it could be flashed even on devices with newer firmware.

Audiotrak-Maya-U5-firmware-mod.zip

Reasonable mouse support in tmux

Yes, we finally got sane, configurable mouse support. In version 2.1 they changed mouse-mode, mouse-select-window/pane etc with single mouse switch. Mouse actions now generates key events that can be mapped as ordinary keys.

In my distro (Ubuntu 14.04) there is version 1.8 of tmux, so we need to get latest from sources:

sudo apt-get build-dep tmux
sudo apt-get clean tmux
wget https://github.com/tmux/tmux/releases/download/2.1/tmux-2.1.tar.gz
tar xzf tmux-2.1.tar.gz
cd tmux-2.1
./configure
make
sudo make install

In manual (man tmux) in paragraph MOUSE SUPPORT we could read that new key events available are named MouseUpX, MouseDownX and MouseDragX where X is button no (1-3), followed by location suffix that describe where you are pointing cursor (Pane, Border or Status). So when you right-click on the status line, events MouseDown3Status and MouseUp3Status will be emitted.

Ok, but how is that better than the former method? You could now define your mouse behaviour as you like. That include (some limited) use of scripting. E.g. to spawn new window after selected by right click on the status line label, you could add something like this to your .tmux.rc:

# don't forget to turn mouse on
set mouse on

bind-key -n MouseDown3Status new-window -a -t=

Option -t= means that the target is window/panel (depends on command) that is clicked.

Or maybe you want to be able to reorder windows in status bar by drag & drop?

bind-key -n MouseDrag1Status swap-window -t=

Ok, that’s great, but we all know what you really want in tmux.

Scroll with mouse in every situation

Yep, it is possible with tmux 2.1. It is not pretty, but it works. And by every situation I mean normal and alternative terminal mode and also tmux copy mode (when you can scroll through history). You could even scroll up to access this mode.

bind-key -n WheelUpPane \
    if-shell -Ft= "#{?pane_in_mode,1,#{mouse_button_flag}}" \
        "send-keys -M" \
        "if-shell -Ft= '#{alternate_on}' \
            'send-keys Up Up Up' \
            'copy-mode'"

bind-key -n WheelDownPane \
    if-shell -Ft= "#{?pane_in_mode,1,#{mouse_button_flag}}" \
        "send-keys -M" \
        "send-keys Down Down Down"

Command if-shell -F is used to check given variable value. If it is non-zero and non empty, first argument will be evaluated, otherwise, second one. Flag pane_in_mode is set if pane is in tmux copy mode. mouse_button_flag is set when running app is actively capturing mouse (like vim). alternate_on is set whenever terminal working in alternate mode (where there is no history to scroll by, like top). If you want to debug these variables, you could print them in status line

set -g status-right 'mouse_btn_flag:#{mouse_button_flag} pane_in_mode:#{pane_in_mode} alt:#{alternate_on}'

Construction like #{?pane_in_mode,1,#{mouse_button_flag}} checks the value of first variable and returns 1 if it is non-zero, second variable value otherwise. It is logical OR constructed with if-shell syntax.

Starting from second example – wheel down. If we are in tmux copy mode or running app want to catch mouse, we send mouse escape strings directly (send-keys -M will pass through mouse events). Otherwise we are sending down arrow key three times. Why not pass mouse event in all cases? Well, if running app don’t tell terminal to catch mouse, most terminals will be doing same thing. That’s why you could scroll through less and man pages.

Wheel up scenario has one more condition added. You can go to copy mode, and then scroll through tmux history when you are not in alternative mode.

This is most sane setup that I’ve been able to come with. It works with shells, vim, man pages, less, htop, mc without breaking terribly anything. One drawback (for me) is, scrolling through tmux copy mode progress by one line at a time. It probably could be fixed by adding extra condition to these lines, but I’m afraid that it will break something. And it is obfuscated enough, already.

For reference, here is my .tmux.conf.

Kernel for NAS ZyXEL NSA310

There are some kernels for this NAS on the web (binaries and configs), but everything I could find was super old (like kernel 3.6.9 old). Because I’ve been able to successfully build current longterm version of Linus tree kernel, here is me sharing my solution.

It is based on this instruction. The kirkwood_defconfig was merged with mvebu_v5_defconfig so we are using the second one instead. I’ve pushed my config, patches and cross-compilation script to my github: https://github.com/sc0ty/nsa310-kernel. There is everything explained in README.md, I won’t repeat myself here. I’ll try to keep this repository up to date with Linus kernel as long as I am using this NAS myself.

And here are binaries (uImage with modules):

Welcome

I’ve finally decided to use some open-source CMS instead coding my own. The reason is that I never had enough time to finish that project. That new version of my site will be regularly updated (I hope so). I will write here about my computer related hobbies: reverse engineering, programming, building electronic devices, etc.

First version of this site was published on Yahoo Geocities many years ago (about 2001 or 2002) and was entirely devoted to my Siemens SL45i patches. I published the whole pack for common good.

Few years later I’ve moved forward, going to Siemens SX1 – next great Siemens with patchable software. About 2006 I’ve moved my site into new server (thanks to MrK and Mac – admins ofp sx1.pl and symbianos.pl), changed whole design and released some SX1 patches (I publish them too).

Now site is located on my private server turned into blog. I hope it will motivate me to write here more often, and maybe to finish some project for which I have no time nor determination to complete.

About me

My real name is Mike Szymaniak and I live in Poznan, Poland. I study IT at Poznan University of Technology. I work soldering circuit boards, I also fix cellphones for living (mainly hardware).

My native language is Polish and I am fully aware that I commit many mistakes writing English, I promise to do my best.

My email: sc0typl[at]gmail.com, GG: 6561102