Downing St brushes off criticism