Status Update
Comments
yb...@google.com <yb...@google.com> #2
ak...@gmail.com <ak...@gmail.com> #3
ah...@gmail.com <ah...@gmail.com> #4
ak...@gmail.com <ak...@gmail.com> #5
ty...@gmail.com <ty...@gmail.com> #6
This request is similar to
Why are we so interested in this?
Our goal is to build an easy to maintain, basically 'NoETL' datawarehouse pipeline. Everywhere where we have to 'fall back' to Python programming makes the whole pipeline more complex to maintain. Having (parametrized) materialized views that could do a full or partial refresh would really be THE enabler for this.
Basically it would be something like Looker's Persistent Direived Tables, but then with the added abilities to:
-Refresh incrementally
-Fully use time partitioning and table suffixing.
yb...@google.com <yb...@google.com>
ra...@gmail.com <ra...@gmail.com> #7
Using aggregation views to optimize query performance and cost is a different use case than using MV as a workaround to ELT.
th...@gmail.com <th...@gmail.com> #8
IMO the main difference between the two patterns are:
1) ETL type MV's have more joins
2) Cost optimization MV's are simpler / aggregate style queries
In the end, what is important is that it supports the following refresh patterns:
1) easy: a full refresh (truncate / insert pattern). Oracle calls this a full refresh
2) more complex: incremental update, for this to work the insert/update/delete (or merge) has to be driven by a key. Oracle calls this the fast refresh option.
IMO this functionality could / should replace BQ scheduled queries, which I guess won't be a real focus anymore now that cloud composer is here...
ro...@gmail.com <ro...@gmail.com> #9
yb...@google.com <yb...@google.com> #10
xm...@gmail.com <xm...@gmail.com> #11
We just saw it in gcloud CLI and would like to try this feature.
al...@gmail.com <al...@gmail.com> #12
mw...@gmail.com <mw...@gmail.com> #13
ak...@gmail.com <ak...@gmail.com> #14
rh...@gmail.com <rh...@gmail.com> #15
ih...@gmail.com <ih...@gmail.com> #16
me...@gmail.com <me...@gmail.com> #17
bq mk \
--transfer_config \
--target_dataset='target_dataset' \
--display_name='schedule_name' \
--params='{"query":"SELECT xyz FROM table","destination_table_name_template":"target_table","write_disposition":"WRITE_APPEND"}' \
--data_source='scheduled_query' \
--schedule='every day 01:00'
da...@google.com <da...@google.com> #18
xm...@gmail.com <xm...@gmail.com> #19
na...@gmail.com <na...@gmail.com> #20
to...@gmail.com <to...@gmail.com> #21
ro...@gmail.com <ro...@gmail.com> #22
lu...@gmail.com <lu...@gmail.com> #23
I have a customer that is very interested in joining early access for this functionality.
Can I help them by signing up somewhere?
Thanks,
Lorin
yb...@google.com <yb...@google.com> #24
gu...@gmail.com <gu...@gmail.com> #25
ki...@gmail.com <ki...@gmail.com> #26
ti...@googlemail.com <ti...@googlemail.com> #27
ur...@gmail.com <ur...@gmail.com> #28
th...@gmail.com <th...@gmail.com> #30
lu...@gmail.com <lu...@gmail.com> #32
Thanks everyone for the suggestions. Materialized Views V2 will be coming soon.
yb...@google.com <yb...@google.com> #33
jo...@gmail.com <jo...@gmail.com> #34
The upcoming version of materialized views will add JOIN support to allow multi-table views as well as table projections.
de...@gmail.com <de...@gmail.com> #36
If I need to create separated data classes for each step and work with any dependent entity manually, creating a tone of boilerplate code - what is a profit to use such ORM, comparing to a default cursors writing?
yb...@google.com <yb...@google.com> #37
if you want to use Room just to be able to have complex objects that are separated to different tables where you can read-write objects, maybe room is not what you want.
Getting objects with relations implicitly
has a cost of getting too much value from database (or doing it lazy has the risk of unexpected queries), we don't want it to be implicit
hence that is why the Relation annotation requires a separate non-entity
class.
You may want to use something like GreenDao or Realm for that. Using cursors will not give you anything similar anyways.
Sorry, we try to find the right balance in Room between avoiding boilerplate code but not hiding the power of sqlite so sometimes it does not fit all expectations.
Description
Version used: 1.0.0-alpha3
I've added one to many relationship in Room using Relation. I am able to fetch the entities while maintaining the relationship.
While storing the entities into the database resulted in `userId` to be empty which means there is no relation between the 2 tables.
1. User Entity (Kotlin):
@Entity
public class User {
@PrimaryKey
public int id; // User id
}
2. Pet Entity (Kotlin):
@Entity
public class Pet {
@PrimaryKey
public int id; // Pet id
public int userId; // User id
public String name;
}
3. UserWithPets POJO (Java):
// Note: No annotation required at this class definition.
public class UserWithPets {
@Embedded
public User user;
@Relation(parentColumn = "id", entityColumn = "userId", entity = Pet.class)
public List<Pet> pets;
}
Now to fetch the records from DB we use the following DAO:
4. UserDao Dao (Kotlin):
@Dao
public interface UserDao {
@Insert
fun insertUser(user: User)
@Query("SELECT * FROM User")
public List<UserWithPets> loadUsersWithPets();
}
5. PetDao Dao (Kotlin):
@Dao
public interface PetDao {
@Insert
void insertAll(List<Pet> pets);
}
Current workaround:
Before inserting `User` and it's list of `Pets`. We need to manually iterate and set the value for `userId` in the `Pet`.
Something like this:
// Java
for(Pet pet : pets) {
pet.setUserId(user.getId());
}
petDao.insertAll(pets);
StackOverFlow Link: