128
read all the rows (a table scan). However, if there is an index on the data, the optimizer merely needs to generate the virtual generated column, which is potentially more efficient.
To mitigate the potential performance issue, we can add a virtual generated column on the table using the voltage element. The following shows the ALTER TABLE statements we can use to add the virtual generated column.
ALTER TABLE `test`.`thermostats` ADD COLUMN voltage INT GENERATED ALWAYS AS (capabilities->'$.voltage') VIRTUAL;
ALTER TABLE `test`.`thermostats` ADD INDEX volts (voltage);
Note If you leave off the option, the generated column generated is a virtual
Listing 3-24. Optimizer EXPLAIN Results for Query
MySQL localhost:33060+ ssl SQL > DROP TABLE IF EXISTS
`test`.`thermostats`;
Query OK, 0 rows affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > CREATE TABLE `test`.`thermostats`
(`model_number` char(20) NOT NULL,`manufacturer` char(30) DEFAULT NULL,`capabilities` json DEFAULT NULL,PRIMARY KEY (`model_number`)) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Query OK, 0 rows affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > INSERT INTO `test`.`thermostats`
VALUES ('ODX-123','Genie','{"rpm": 3000, "color": "white", "modes": ["ac",
"furnace"], "voltage": 220, "capability": "fan"}');
Query OK, 1 row affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > INSERT INTO `test`.`thermostats` VALUES ('AB-90125-C1', 'Jasper', '{"rpm": 1500, "color": "beige", "modes": ["ac"],
"voltage": 110, "capability": "auto fan"}');
Query OK, 1 row affected (0.00 sec)
# Query without virtual generated column.
MySQL localhost:33060+ ssl SQL > EXPLAIN SELECT * FROM thermostats WHERE capabilities->'$.voltage' = 110 \G
*************************** 1. row ***************************
id: 1 select_type: SIMPLE table: thermostats partitions: NULL
type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 23302
130
filtered: 100.00 Extra: Using where
1 row in set, 1 warning (0.00 sec)
Note (code 1003): /* select#1 */ select `test`.`thermostats`.`model_
number` AS `model_number`,`test`.`thermostats`.`manufacturer` AS `man ufacturer`,`test`.`thermostats`.`capabilities` AS `capabilities` from
`test`.`thermostats` where (json_extract(`test`.`thermostats`.`capabilities`, '$.voltage') = 110)
MySQL localhost:33060+ ssl SQL > ALTER TABLE `test`.`thermostats`
ADD COLUMN color char(20) GENERATED ALWAYS AS (capabilities->'$.color') VIRTUAL;
Query OK, 0 rows affected (0.00 sec)
# Query with virtual generated column.
MySQL localhost:33060+ ssl SQL > DROP TABLE `test`.`thermostats`;
Query OK, 0 rows affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > CREATE TABLE `thermostats` (`model_
number` char(20) NOT NULL, `manufacturer` char(30) DEFAULT NULL,
`capabilities` json DEFAULT NULL, `voltage` int(11) GENERATED ALWAYS AS (json_extract(`capabilities`,'$.voltage')) VIRTUAL, PRIMARY KEY (`model_
number`), KEY `volts` (`voltage`)) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Query OK, 0 rows affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > EXPLAIN SELECT * FROM thermostats WHERE capabilities->'$.voltage' = 110 \G
*************************** 1. row ***************************
id: 1 select_type: SIMPLE table: thermostats partitions: NULL
type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL Chapter 3 JSON DOCumeNtS
rows: 1102 filtered: 100.00 Extra: Using where
1 row in set, 1 warning (0.00 sec)
Note (code 1003): /* select#1 */ select `test`.`thermostats`.`model_number`
AS `model_number`,`test`.`thermostats`.`manufacturer` AS `manufacturer`,
`test`.`thermostats`.`capabilities` AS `capabilities`,`test`.`thermostats`.
`color` AS `color` from `test`.`thermostats` where (json_extract(`test`.
`thermostats`.`capabilities`,'$.voltage') = 110)
Note that the first EXPLAIN shows no use of an index (no key, key_len) whereas the second does show the use of an index. The rows result shows how many rows (estimated) will be read to make the comparison. It is clear that adding a generated column and an index can help us optimize our queries of JSON data in relational tables. Cool.
However, there is one thing the example did not cover. If the JSON data element is a string, you must use the JSON_UNQUOTE() function to remove the quotes from the string.
Let’s suppose we wanted to add a generated column for the color data element. If we add the column and index with the ALTER TABLE statements without removing the quotes, we will get some unusual results as shown in Listing 3-25.
Listing 3-25. Removing Quotes for Generated Columns on JSON Strings
MySQL localhost:33060+ ssl SQL > DROP TABLE IF EXISTS `test`.`thermostats`;
Query OK, 0 rows affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > CREATE TABLE `test`.`thermostats`
(`model_number` char(20) NOT NULL,`manufacturer` char(30) DEFAULT NULL,`capabilities` json DEFAULT NULL,PRIMARY KEY (`model_number`)) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Query OK, 0 rows affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > INSERT INTO `test`.`thermostats`
VALUES ('ODX-123','Genie','{"rpm": 3000, "color": "white", "modes": ["ac",
"furnace"], "voltage": 220, "capability": "fan"}');
Query OK, 1 row affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > INSERT INTO `test`.`thermostats` VALUES ('AB-90125-C1', 'Jasper', '{"rpm": 1500, "color": "beige", "modes": ["ac"],
"voltage": 110, "capability": "auto fan"}');
Query OK, 1 row affected (0.00 sec)
132
MySQL localhost:33060+ ssl SQL > ALTER TABLE `test`.`thermostats`
ADD COLUMN color char(20) GENERATED ALWAYS AS (capabilities->'$.color') VIRTUAL;
Query OK, 0 rows affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > SELECT model_number, color FROM thermostats WHERE color = "beige";
Empty set (0.00 sec)
MySQL localhost:33060+ ssl SQL > SELECT model_number, color FROM thermostats LIMIT 2;
+---+---+
| model_number | color | +---+---+
| AB-90125-C1 | "beige" |
| ODX-123 | "white" | +---+---+
2 rows in set (0.00 sec)
MySQL localhost:33060+ ssl SQL > ALTER TABLE thermostats DROP COLUMN color;
Query OK, 0 rows affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > ALTER TABLE thermostats ADD COLUMN color char(20) GENERATED ALWAYS AS (JSON_UNQUOTE(capabilities->'$.color')) VIRTUAL;
Query OK, 0 rows affected (0.00 sec)
MySQL localhost:33060+ ssl SQL > SELECT model_number, color FROM thermostats WHERE color = 'beige' LIMIT 1;
+---+---+
| model_number | color | +---+---+
| AB-90125-C1 | beige | +---+---+
1 row in set (0.00 sec)
Note that in the first SELECT statement, there is nothing returned. This is because the virtual generated column used the JSON string with the quotes. This is often a source of confusion when mixing SQL and JSON data. Note that in the second SELECT statement, we see there should have been several rows returned. Note also that after I dropped the column and added it again with the JSON_UNQUOTE() function, the SELECT returns the correct data.
Chapter 3 JSON DOCumeNtS
We normally use a virtual generated column so that we don’t store anything extra in the row. This is partly because we may not use the index on the JSON data very often and may not need it maintained, but more important because there are restrictions on how you can use/define a stored generated column. The following summarize the restrictions.
• The table must have a primary key defined.
• You must use either a FULLTEXT or RTREE index (instead of the default BTREE).
However, if you have a lot of rows or are using the index on the JSON data frequently or have more than one index on the JSON data, you may want to consider using the stored generated column because virtual generated columns can be computationally taxing when accessing complex or deeply nested data frequently.