An xtabs object is indeed already a matrix, but won't be converted to a pure matrix by as.matrix.default function, as its class definition will remain unchanged. Some functions expecting a pure matrix may fail, when fed with a xtabs object.
as.matrix.xtabs will drop the classes and the call attribute.
Note that unclass would as well discard the classes xtabs and table, but retain the "call" attribute.