You might want to read about the Cyberoam Protocol before anything on this page will make sense to you. As you will discover on that page, I have successfully reverse engineered the password encryption algorithm, and have posted it here.

My Guesswork Involving the Password Encryption Algorithm

Here are my thoughts as I entered them on the protocol document, before I had successfully determined the algorithm for encrypting the password.

• The length of the encrypted password is (10 + 3L), where L is the length of the unencrypted password. It appears that the individual characters of the password are encoded into 3-digit integers, whereas the first 10 digits form some sort of time stamp, or base value from which to compute the 3-digit integers.
• I can confirm the first 10 digits are a count of the number of seconds elapsed since January 1, 1970, 00:00:00 UTC (i.e. at 00:00:00 GMT). So, for example, on Thursday, July 18, 2002, at approximately 05:45:00, the value of the 10-digit field was "1026951299". This is the same value returned by the time() function in the standard ANSI C Library. One of the well known uses of the time() function is to generate a seed value for the random number generator function, srand(). If the algorithm uses rand(), this needs to be investigated.
• The 3-digit integers range from 0-255 (the maximum value detected by me so far is 252), so I am guessing that the ASCII code for each character of the original password is encrypted to another ASCII character, with the resulting values inserted in this field. However, "aa" does not produce the same encrypted values for each character, so there is some sort of hash algorithm or other somewhat complicated algorithm being applied.
• It is somewhat silly that the official Cyberoam client only allows a maximum length of 8 characters for the password, whereas their protocol reserves 60 bytes for the password. 8 characters only require (10+8*3=34) bytes, hence 24 bytes are wasted.
• The algorithm probably doesn't use RSA, because none of the 3 digit integer groupings have a value larger than 255. RSA works by choosing an integer n such that n=(p*q) where p and q are prime numbers. But n can not be larger than 255, so this would only allow for n=253 (p=23,q=11) and 254 (p=2,q=127) as potential RSA keys, but the size of the time stamp (10 digits) means this would be a huge mismatch.

I was able to bypass the call to time() in the encryption subroutine after disassembling the client binary. Further, after attempting to analyze the disassembled code, I learned a little more about the algorithm. Let p[n] be the n'th byte of the plaintext password, and c[n] the n'th byte of the ciphertext, with n ranging from 0..7(max). Then:

1. Changing p[0] changes all the bytes of c.
2. Changing p[i] changes only bytes c[i..n] of the ciphertext, i.e. only subsequent bytes in a password are affected.
3. c[0] depends only on p[0] and the timestamp.
4. c[i] depends on c[i-1], p[i] and the timestamp, for i=1..n.
5. The timestamp is passed through a hashing algorithm for each character in the password. The hashing algorithm produces a 2-byte digest, which is then combined with p[i] as follows: c[i] := (lo xor p[i]) xor hi where hi and lo are the high order and low order bytes of the 2-byte digest, respectively.
6. I have still not worked out the exact digest algorithm, but let's see what happens in the next few days.
7. The plaintext is encrypted incrementally, with each increment performing a repeat computation of the 2-byte digest for the timestamp.

I know the following so far about the computation of the timestamp digest:

1. 2 bytes of the timestamp (after converting to a 10-digit ASCII string) are hashed at a time, with subsequent pairs of bytes using results from the directly previous pair.
2. The hash algorithm involves multiplication by the constants 0x4E35 (20021) and 0x015A (346) for each pair of bytes.
3. The hash value for each pair of bytes depends only on the hash value for the previous pair of bytes.

After the above deductions, I was successfully able to determine the actual algorithm on 13.Aug.2002.