How do I combine two keras generator functions

发布于 2021-01-29 14:09:45

I am trying to implement a Siamese network in Keras and I want to apply image
transformations to the 2 input images using Keras Image Data Generators. As
per the example in the docs- https://keras.io/preprocessing/image/, I’ve
tried to implement it like this-

datagen_args = dict(rotation_range=10,
                    width_shift_range=0.1,
                    height_shift_range=0.1,
                    horizontal_flip=True)

in_gen1 = ImageDataGenerator(**datagen_args)
in_gen2 = ImageDataGenerator(**datagen_args)

train_generator = zip(in_gen1, in_gen2)

model.fit(train_generator.flow([pair_df[:, 0,::],pair_df[:, 1,::]],
                          y_train,batch_size=16), epochs, verbose = 1)

But this code throws this error:

TypeError : zip argument #1 must support iteration

I’ve tried using itertools.izip as suggested in Keras - Generator for large
dataset of Images and
Masks
but this throws the same error.

How do I resolve this?

EDIT : In case anyone is interested, this worked finally-

datagen_args = dict(
    featurewise_center=False,
    rotation_range=10,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True)

in_gen1 = ImageDataGenerator(**datagen_args)
in_gen2 = ImageDataGenerator(**datagen_args)

in_gen1 = in_gen1.flow(pair_df[:, 0,::], y_train, batch_size = 16, shuffle = False)
in_gen2 = in_gen2.flow(pair_df[:, 1,::], y_train, batch_size = 16, shuffle = False)

for e in range(epochs):
    batches = 0
    for x1, x2 in itertools.izip(in_gen1,in_gen2):
    # x1, x2 are tuples returned by the generator, check whether targets match
        assert sum(x1[1] != x2[1]) == 0  
        model.fit([x1[0], x2[0]], x1[1], verbose = 1)
        batches +=1
        if(batches >= len(pair_df)/16):
            break
关注者
0
被浏览
231
1 个回答
  • 面试哥
    面试哥 2021-01-29
    为面试而生,有面试问题,就找面试哥。

    Using zip() to combine generators leads to generation of an infinite
    iterator. Use this instead:

    def combine_generator(gen1, gen2):
        while True:
            yield(next(gen1), next(gen2))
    

    Modified code would look something like this:

    datagen_args = dict(rotation_range=10,
                        width_shift_range=0.1,
                        height_shift_range=0.1,
                        horizontal_flip=True)
    
    in_gen1 = ImageDataGenerator(**datagen_args)
    in_gen2 = ImageDataGenerator(**datagen_args)
    
    def combine_generator(gen1, gen2):
        while True:
            yield(next(gen1), next(gen2))
    
    train_generator = combine_generator(in_gen1, in_gen2)
    
    model.fit(train_generator.flow([pair_df[:, 0,::],pair_df[:, 1,::]],
                              y_train,batch_size=16), epochs, verbose = 1)
    

    See this thread for further
    reference.



知识点
面圈网VIP题库

面圈网VIP题库全新上线,海量真题题库资源。 90大类考试,超10万份考试真题开放下载啦

去下载看看