Tuesday, February 26, 2008

Passwords in phpBB 3

Here is a port of phpBB3's password handling to Java.

import java.io.UnsupportedEncodingException;
import java.security.GeneralSecurityException;
import java.security.MessageDigest;

/**
* Port of phpBB3 password handling to Java.
* See phpBB3/includes/functions.php
*
* @author lars
*/
public class PHPBB3Password {
private static final int PHP_VERSION = 4;
private String itoa64 =
"./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";

public String phpbb_hash(String password) {
String random_state = unique_id();
String random = "";
int count = 6;

if (random.length() < count) {
random = "";

for (int i = 0; i < count; i += 16) {
random_state = md5(unique_id() + random_state);
random += pack(md5(random_state));
}
random = random.substring(0, count);
}

String hash = _hash_crypt_private(
password, _hash_gensalt_private(random, itoa64));
if (hash.length() == 34)
return hash;

return md5(password);
}

private String unique_id() {
return unique_id("c");
}

// global $config;
// private boolean dss_seeded = false;

private String unique_id(String extra) {
// TODO Generate something random here.
return "1234567890abcdef";
}

private String _hash_gensalt_private(String input, String itoa64) {
return _hash_gensalt_private(input, itoa64, 6);
}

private String _hash_gensalt_private(
String input, String itoa64, int iteration_count_log2) {
if (iteration_count_log2 < 4 || iteration_count_log2 > 31) {
iteration_count_log2 = 8;
}

String output = "$H$";
output += itoa64.charAt(
Math.min(iteration_count_log2 +
((PHP_VERSION >= 5) ? 5 : 3), 30));
output += _hash_encode64(input, 6);

return output;
}

/**
* Encode hash
*/
private String _hash_encode64(String input, int count) {
String output = "";
int i = 0;

do {
int value = input.charAt(i++);
output += itoa64.charAt(value & 0x3f);

if (i < count)
value |= input.charAt(i) << 8;

output += itoa64.charAt((value >> 6) & 0x3f);

if (i++ >= count)
break;

if (i < count)
value |= input.charAt(i) << 16;

output += itoa64.charAt((value >> 12) & 0x3f);

if (i++ >= count)
break;

output += itoa64.charAt((value >> 18) & 0x3f);
} while (i < count);

return output;
}

String _hash_crypt_private(String password, String setting) {
String output = "*";

// Check for correct hash
if (!setting.substring(0, 3).equals("$H$"))
return output;

int count_log2 = itoa64.indexOf(setting.charAt(3));
if (count_log2 < 7 || count_log2 > 30)
return output;

int count = 1 << count_log2;
String salt = setting.substring(4, 12);
if (salt.length() != 8)
return output;

String m1 = md5(salt + password);
String hash = pack(m1);
do {
hash = pack(md5(hash + password));
} while (--count > 0);

output = setting.substring(0, 12);
output += _hash_encode64(hash, 16);

return output;
}

public boolean phpbb_check_hash(
String password, String hash) {
if (hash.length() == 34)
return _hash_crypt_private(password, hash).equals(hash);
else
return md5(password).equals(hash);
}

public static String md5(String data) {
try {
byte[] bytes = data.getBytes("ISO-8859-1");
MessageDigest md5er = MessageDigest.getInstance("MD5");
byte[] hash = md5er.digest(bytes);
return bytes2hex(hash);
} catch (GeneralSecurityException e) {
throw new RuntimeException(e);
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(e);
}
}

static int hexToInt(char ch) {
if(ch >= '0' && ch <= '9')
return ch - '0';

ch = Character.toUpperCase(ch);
if(ch >= 'A' && ch <= 'F')
return ch - 'A' + 0xA;

throw new IllegalArgumentException("Not a hex character: " + ch);
}

private static String bytes2hex(byte[] bytes) {
StringBuffer r = new StringBuffer(32);
for (int i = 0; i < bytes.length; i++) {
String x = Integer.toHexString(bytes[i] & 0xff);
if (x.length() < 2)
r.append("0");
r.append(x);
}
return r.toString();
}

static String pack(String hex) {
StringBuffer buf = new StringBuffer();
for(int i = 0; i < hex.length(); i += 2) {
char c1 = hex.charAt(i);
char c2 = hex.charAt(i+1);
char packed = (char) (hexToInt(c1) * 16 + hexToInt(c2));
buf.append(packed);
}
return buf.toString();
}
}

Friday, February 22, 2008

Agile practices don't primarily solve problems

But they sure help detecting problems early. And that's extremely valuable.

For instance, iterative development with deliveries early in the project will test the technical feasibility and the development speed. But perhaps most important, we will get feedback on the functionality from the clients, to verify if we have understood their problem correctly.

Failing in any of these areas will lead to project failure. And it is better to discover this when 20% of the budget is used than when 90% of the budget is used. First, you reduce your losses. And even better, you have a fair chance of fixing the project and turning it into success.

Many agile projects succeed not because they are more productive, but because they discover problems in early iteations and then reduce the requirements.

Unit testing is another practice that makes you aware of problems early. This makes it cheaper to fix problems, but it also gives confidence, so you dear restructure code when necessary.

Another agile practice is close communication with business experts, preferably face to face. The non-agile alternative is written requirements. The problem is that developers misunderstand what the business experts want. That's inevitable. What we need, are mechanisms to discover misunderstandings. That mechanism is called feedback. The business expert explain what they want, and the developer explain how they understood that to the business expert, which confirms that this was correct. That's easier to do face to face than with written documents.

Monday, February 11, 2008

Hungarian notation and Groovy

Comments and types are metadata and strictly not necessary to make a program, according to a popular post. Yes, they are not necessary to communicate with the computer, but that's not what we are trying to do here. For all but the smallest projects, we are communicating with people. The code should be understandable to others. And it doesn't hurt if it's understandable to ourselves either.

Hungarian notation was a popular in C to show the type of a variable in its name, for instance:
int nSize;
char cChar;

I never liked this, because if I change the type of a variable I would also have to change its name everywhere. But with dynamically typed languages, I some times get in trouble because I forget what type a variable is. So, I started to think if maybe Hungarian notation could be useful.

For example, consider the following dynamically typed Groovy code:
(1) def x = "100"

the following statically typed Java code:
(2) String x = "100";

and this statically typed Java code:
(3) int x = "100";

Which is most easy to understand?
(3) is easily recognized by both humans and the compiler as an error.
(1) and (2) are valid code. But 20 lines down the code, you have forgotten what type x is and write:
x++

Then (2) will give you a compiler error, but (1) won't be detected until runtime.

Does this mean I'm against dynamically typing? No. But if we remove the static type checking, I think we should use other means to communicate that information. For instance, it is a bad idea to call a String x. This would be much better:
def s = "100"

In Groovy, I often use maps instead of creating a new class:
def map = [:]
map.x = 100
map.y = 200

How am I supposed to remember what the map contains down the road?

I don't want to remember, I want the map to remind me! I have enough things to think about, I don't want to remember the type of every variable too!

I have a lame technique I use to not forget important things: I try to make them remind me instead of having to remember them. For instance, if I take off the lid to fill gas in the car, I put the car keys by the lid. That way, it's impossible for me to forget it. I cannot drive off without seeing the lid and remember to put it on. I use similar techniques at work. This reduces stress and helps me to concentrate, because I don't need to remember so much.

Applying this technique to Groovy gives this code:
def point = [:]
point.x = 100
point.y = 200

So, I am not advocating Hungarian notation, but good naming conventions that communicate what we need to know.

Programming is more about communication than how fast you can crank out code.

Wednesday, February 6, 2008

The Pessimistic Programmer

I decided to change the title of this blog to "The Pessimistic Programmer". Why? Am I a depressed person that thinks nothing will work? No, I am an optimist in life. Something good is going to happen today :-) But in programming, something will surely go wrong.

I don't actually view this as pessimism, but as realism. I want to be prepared for the worst that can possibly happen. Hope for the best, prepare for the worst. But my wife explained to me that pessimists always say that they are just being realistic. So, I might as well face it: I am a pessimist.

I think a good programmer needs to be pessimistic; always thinking about what can go wrong and how to prevent it. I don't say that I am a good programmer myself. No, I make far too many mistakes for that. But I have learnt how to manage my mistakes with testing and double checking.

Über-programmers can manage well without being pessimistic. They have total overview of the code and all consequences of changes. But I'm talking about us mere mortals. But if you are an über-programmer, you should be pessimistic about what the next guy will do to your code. Place some comments and unit tests in there, to keep him out of trouble!

An optimistic programmer doesn't see the need for unit tests. He will run the program and be satisfied when it runs one time without errors. He will say things like "What can possibly go wrong?" Or if something goes wrong anyway, "It must be an error in the input from that other module".

Haven't we all been there? Suddenly, I feel so old...

Passwords in SMF 1.1.4

I'm posting this in case someone else are struggling with SMF password encrypting like I did.

SMF 1.1.4 uses SHA-1 with a salt. You would think that the passwordSalt in the database is used as the salt, but it isn't. That's probably a field that was used in old versions. Instead, the membername is used as the salt, but *before* the password, not *after*, as most search results indicated.

I finally found this link to a php file that is *not* in the distribution.
function smf_registerMember(
[...]
$register_vars = array(
'memberName' => "'$username'",
'realName' => "'$username'",
'passwd' => '\'' . sha1(strtolower($username) . $password) . '\'',
'passwordSalt' => '\'' . substr(md5(rand()), 0, 4) . '\'',


So I tried this Java code:
sha1(username.toLowerCase() + password);
That worked!

Here is the source for the sha1 method:
   public static String sha1(String data)
{
byte[] bytes = data.getBytes();
try
{
MessageDigest md5er = MessageDigest.getInstance("SHA-1");
byte[] hash = md5er.digest(bytes);
return bytes2hex(hash);
}
catch (GeneralSecurityException e)
{
throw new RuntimeException(e);
}
}

private static String bytes2hex(byte[] bytes)
{
StringBuffer r = new StringBuffer(32);
for (int i = 0; i < bytes.length; i++)
{
String x = Integer.toHexString(bytes[i] & 0xff);
if (x.length() < 2)
r.append("0");
r.append(x);
}
return r.toString();
}