As of writing this, have found a problem with using Barracuda table format in Aurora.
Per the MySQL manual, to use Barracuda, you can set ROW_FORMAT=COMPRESSED or ROW_FORMAT=DYNAMIC.
From: https://dev.mysql.com/doc/refman/5.6/en/innodb-compression-usage.html
Aurora doesn’t actually support compressed tables and will automatically change the format if you try to use ROW_FORMAT=COMPRESSED.
Per their documentation:
Amazon Aurora doesn't support compressed tables (that is, tables created with ROW_FORMAT=COMPRESSED).
Copied from: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/AuroraMySQL.Migrating.RDSMySQL.Import.html
So, if you want to use Barracuda, you are left with ROW_FORMAT=DYNAMIC. I have a client who wanted to use Barracuda and was using it on a physical machine at their data center. After doing the migration to Aurora, everything automatically changed to Antelope because the table create statements were using ROW_FORMAT=COMPRESSED. We went through and changed everything to ROW_FORMAT=DYNAMIC in order to force the tables to use Barracuda. This client has a sharded application spread out over about 30 instances. Each cluster has a writer end point instance and a replica for read only traffic. After we converted the tables to Barracuda, the read only replicas would randomly restart. The error on the application side was "org.mariadb.jdbc.internal.util.dao.QueryException: unexpected end of stream". Then they would get a number of errors related to the instance not being available. Then it would work fine for a while.
It was hard to figure it out but eventually we traced it back to the read only replicas using Barracuda. For a while we transferred all the read only traffic to the writer end points and all the problems stopped.
If I log onto the writer end point, you can see that I've set the innodb_file_format and innodb_file_format_max to Barracuda (same settings for read only replica also).
mysql> show global variables like 'innodb_file_%';
+--------------------------+-----------+
| Variable_name | Value |
+--------------------------+-----------+
| innodb_file_format | Barracuda |
| innodb_file_format_check | ON |
| innodb_file_format_max | Barracuda |
| innodb_file_per_table | ON |
+--------------------------+-----------+
4 rows in set (0.09 sec)
This query shows I'm on the writer end point:
mysql> show global variables like 'innodb_read_only';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| innodb_read_only | OFF |
+------------------+-------+
1 row in set (0.12 sec)
mysql> show global variables like 'aurora_version';
+----------------+--------+
| Variable_name | Value |
+----------------+--------+
| aurora_version | 1.15.1 |
+----------------+--------+
1 row in set (0.12 sec)
mysql>
mysql> use tmp;
Database changed
mysql>
mysql> DROP TABLE IF EXISTS test_table;
Query OK, 0 rows affected (0.12 sec)
mysql> CREATE TABLE `test_table` (
-> `id` int(11) NOT NULL AUTO_INCREMENT,
-> `message` varchar(255) NOT NULL,
-> `created_at` datetime NOT NULL,
-> PRIMARY KEY (`id`)
-> ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 ROW_FORMAT=COMPRESSED;
Query OK, 0 rows affected, 2 warnings (0.13 sec)
mysql> SELECT * FROM information_schema.INNODB_SYS_TABLES WHERE NAME = 'tmp/test_table';
+----------+----------------+------+--------+-------+-------------+------------+---------------+
| TABLE_ID | NAME | FLAG | N_COLS | SPACE | FILE_FORMAT | ROW_FORMAT | ZIP_PAGE_SIZE |
+----------+----------------+------+--------+-------+-------------+------------+---------------+
| 196 | tmp/test_table | 1 | 6 | 138 | Antelope | Compact | 0 |
+----------+----------------+------+--------+-------+-------------+------------+---------------+
1 row in set (0.09 sec)
Table became Antelope (Aurora silently changes the format for you).
mysql> DROP TABLE IF EXISTS test_table;
Query OK, 0 rows affected (0.11 sec)
mysql> CREATE TABLE `test_table` (
-> `id` int(11) NOT NULL AUTO_INCREMENT,
-> `message` varchar(255) NOT NULL,
-> `created_at` datetime NOT NULL,
-> PRIMARY KEY (`id`)
-> ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 ROW_FORMAT=DYNAMIC ;
Query OK, 0 rows affected (0.11 sec)
mysql> SELECT * FROM information_schema.INNODB_SYS_TABLES WHERE NAME = 'tmp/test_table';
+----------+----------------+------+--------+-------+-------------+------------+---------------+
| TABLE_ID | NAME | FLAG | N_COLS | SPACE | FILE_FORMAT | ROW_FORMAT | ZIP_PAGE_SIZE |
+----------+----------------+------+--------+-------+-------------+------------+---------------+
| 197 | tmp/test_table | 33 | 6 | 139 | Barracuda | Dynamic | 0 |
+----------+----------------+------+--------+-------+-------------+------------+---------------+
1 row in set (0.10 sec)
Now the table is Barracuda.
The issue we seem to have is something to with Aurora starting up with the setting in the parameter group for innodb_file_format_max as Barracuda. The system tables are using Antelope but when it reads a table using Barracuda, it tries to set the innodb_file_format_max to Barracuda but since it is read only it crashes.
After the read only replica crashes, it seems to fix itself for a while. And then eventually crashes again. I'm not sure how it gets back into a bad state which allows it to crash again. Reverting all the tables back to Antelope solved the issue.