How to add a second input argument (the first is an image) to a CNN model built with Keras?
Let's say I have a list of images (converted to numpy arrays) downloaded from Instagram, along with their corresponding likes and user followers. And let's say I have a CNN model (using Keras on Tensorflow) which I train on these images (200x200x3 numpy arrays) and it tries to predict the number of likes an image will get.
What if I want to give to this model each image's corresponding followers as a second input?
This is my code so far:
IMAGESIZE = (200, 200)
def create_model():
# create model and add layers
model = Sequential()
model.add(Conv2D(10, 5, 5, activation='relu',
input_shape=(IMAGESIZE[0], IMAGESIZE[1], 3)))
model.add(Conv2D(10, 5, 5, activation='relu'))
model.add(MaxPool2D((5, 5)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(50))
model.add(Activation('relu'))
model.add(Dense(1))
print(model.summary())
model.compile(loss='mse',
optimizer='rmsprop', metrics=["accuracy"])
return model
# Read the likes
likes = getlikes(src='../data/pickledump')
likesraw = np.array(likes)
likes = (likesraw - np.mean(likesraw))/np.std(likesraw) # normalize
# Read the images and resize them
images =
for imgfile in glob.glob('../data/download/*.jpeg'):
img = cv2.imread(imgfile)
resized = cv2.resize(img, IMAGESIZE)
images.append(resized)
break
images = np.array(images)
# Read the followers
followers= getfollowers(src='../data/pickledump')
followersraw= np.array(followers)
followers= (followersraw- np.mean(followersraw))/np.std(followersraw) # normalize
classifier = KerasClassifier(build_fn=create_model, epochs=20)
print("Accuracy (Cross Validation=10): ",
np.mean(cross_val_score(classifier, images, likes, cv=2)))
python tensorflow machine-learning keras conv-neural-network
add a comment |
Let's say I have a list of images (converted to numpy arrays) downloaded from Instagram, along with their corresponding likes and user followers. And let's say I have a CNN model (using Keras on Tensorflow) which I train on these images (200x200x3 numpy arrays) and it tries to predict the number of likes an image will get.
What if I want to give to this model each image's corresponding followers as a second input?
This is my code so far:
IMAGESIZE = (200, 200)
def create_model():
# create model and add layers
model = Sequential()
model.add(Conv2D(10, 5, 5, activation='relu',
input_shape=(IMAGESIZE[0], IMAGESIZE[1], 3)))
model.add(Conv2D(10, 5, 5, activation='relu'))
model.add(MaxPool2D((5, 5)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(50))
model.add(Activation('relu'))
model.add(Dense(1))
print(model.summary())
model.compile(loss='mse',
optimizer='rmsprop', metrics=["accuracy"])
return model
# Read the likes
likes = getlikes(src='../data/pickledump')
likesraw = np.array(likes)
likes = (likesraw - np.mean(likesraw))/np.std(likesraw) # normalize
# Read the images and resize them
images =
for imgfile in glob.glob('../data/download/*.jpeg'):
img = cv2.imread(imgfile)
resized = cv2.resize(img, IMAGESIZE)
images.append(resized)
break
images = np.array(images)
# Read the followers
followers= getfollowers(src='../data/pickledump')
followersraw= np.array(followers)
followers= (followersraw- np.mean(followersraw))/np.std(followersraw) # normalize
classifier = KerasClassifier(build_fn=create_model, epochs=20)
print("Accuracy (Cross Validation=10): ",
np.mean(cross_val_score(classifier, images, likes, cv=2)))
python tensorflow machine-learning keras conv-neural-network
add a comment |
Let's say I have a list of images (converted to numpy arrays) downloaded from Instagram, along with their corresponding likes and user followers. And let's say I have a CNN model (using Keras on Tensorflow) which I train on these images (200x200x3 numpy arrays) and it tries to predict the number of likes an image will get.
What if I want to give to this model each image's corresponding followers as a second input?
This is my code so far:
IMAGESIZE = (200, 200)
def create_model():
# create model and add layers
model = Sequential()
model.add(Conv2D(10, 5, 5, activation='relu',
input_shape=(IMAGESIZE[0], IMAGESIZE[1], 3)))
model.add(Conv2D(10, 5, 5, activation='relu'))
model.add(MaxPool2D((5, 5)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(50))
model.add(Activation('relu'))
model.add(Dense(1))
print(model.summary())
model.compile(loss='mse',
optimizer='rmsprop', metrics=["accuracy"])
return model
# Read the likes
likes = getlikes(src='../data/pickledump')
likesraw = np.array(likes)
likes = (likesraw - np.mean(likesraw))/np.std(likesraw) # normalize
# Read the images and resize them
images =
for imgfile in glob.glob('../data/download/*.jpeg'):
img = cv2.imread(imgfile)
resized = cv2.resize(img, IMAGESIZE)
images.append(resized)
break
images = np.array(images)
# Read the followers
followers= getfollowers(src='../data/pickledump')
followersraw= np.array(followers)
followers= (followersraw- np.mean(followersraw))/np.std(followersraw) # normalize
classifier = KerasClassifier(build_fn=create_model, epochs=20)
print("Accuracy (Cross Validation=10): ",
np.mean(cross_val_score(classifier, images, likes, cv=2)))
python tensorflow machine-learning keras conv-neural-network
Let's say I have a list of images (converted to numpy arrays) downloaded from Instagram, along with their corresponding likes and user followers. And let's say I have a CNN model (using Keras on Tensorflow) which I train on these images (200x200x3 numpy arrays) and it tries to predict the number of likes an image will get.
What if I want to give to this model each image's corresponding followers as a second input?
This is my code so far:
IMAGESIZE = (200, 200)
def create_model():
# create model and add layers
model = Sequential()
model.add(Conv2D(10, 5, 5, activation='relu',
input_shape=(IMAGESIZE[0], IMAGESIZE[1], 3)))
model.add(Conv2D(10, 5, 5, activation='relu'))
model.add(MaxPool2D((5, 5)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(50))
model.add(Activation('relu'))
model.add(Dense(1))
print(model.summary())
model.compile(loss='mse',
optimizer='rmsprop', metrics=["accuracy"])
return model
# Read the likes
likes = getlikes(src='../data/pickledump')
likesraw = np.array(likes)
likes = (likesraw - np.mean(likesraw))/np.std(likesraw) # normalize
# Read the images and resize them
images =
for imgfile in glob.glob('../data/download/*.jpeg'):
img = cv2.imread(imgfile)
resized = cv2.resize(img, IMAGESIZE)
images.append(resized)
break
images = np.array(images)
# Read the followers
followers= getfollowers(src='../data/pickledump')
followersraw= np.array(followers)
followers= (followersraw- np.mean(followersraw))/np.std(followersraw) # normalize
classifier = KerasClassifier(build_fn=create_model, epochs=20)
print("Accuracy (Cross Validation=10): ",
np.mean(cross_val_score(classifier, images, likes, cv=2)))
python tensorflow machine-learning keras conv-neural-network
python tensorflow machine-learning keras conv-neural-network
edited Nov 21 '18 at 7:44
today
11.2k22239
11.2k22239
asked Nov 20 '18 at 20:17
drkostasdrkostas
163319
163319
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
One approach is to use a two branch model, where one branch processes the image and another branch processes other non-image inputs (such as posts texts or number of followers and followings, etc.). Then you can merge the result of these two branches and possibly add a few other layers afterwards to act a the final classifier/regressor. To build such a model in Keras you need to use the functional API instead. Just for demonstration, here is an example:
inp_img = Input(shape=image_shape)
inp_others = Input(shape=others_shape)
# branch 1: process input image
x = Conv2D(...)(inp_img)
x = Conv2D(...)(x)
x = MaxPool2D(...)(x)
out_b1 = Flatten()(x)
# branch 2: process other input
out_b2 = Dense(...)(inp_other)
# merge the results by concatenation
merged = concatenate([out_b1, out_b2])
# pass merged tensor to some other layers
x = Dense(...)(merged)
output = Dense(...)(x)
# build the model and compile it
model = Model([inp_img, inp_other], output)
model.compile(...)
# fit on training data
model.fit([img_array, other_array], label_array, ...)
Note that we used concatenation
layer above, but there are other merge layers which you can use. And make sure you read the functional API guide, it's a must-read guide.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400895%2fhow-to-add-a-second-input-argument-the-first-is-an-image-to-a-cnn-model-built%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
One approach is to use a two branch model, where one branch processes the image and another branch processes other non-image inputs (such as posts texts or number of followers and followings, etc.). Then you can merge the result of these two branches and possibly add a few other layers afterwards to act a the final classifier/regressor. To build such a model in Keras you need to use the functional API instead. Just for demonstration, here is an example:
inp_img = Input(shape=image_shape)
inp_others = Input(shape=others_shape)
# branch 1: process input image
x = Conv2D(...)(inp_img)
x = Conv2D(...)(x)
x = MaxPool2D(...)(x)
out_b1 = Flatten()(x)
# branch 2: process other input
out_b2 = Dense(...)(inp_other)
# merge the results by concatenation
merged = concatenate([out_b1, out_b2])
# pass merged tensor to some other layers
x = Dense(...)(merged)
output = Dense(...)(x)
# build the model and compile it
model = Model([inp_img, inp_other], output)
model.compile(...)
# fit on training data
model.fit([img_array, other_array], label_array, ...)
Note that we used concatenation
layer above, but there are other merge layers which you can use. And make sure you read the functional API guide, it's a must-read guide.
add a comment |
One approach is to use a two branch model, where one branch processes the image and another branch processes other non-image inputs (such as posts texts or number of followers and followings, etc.). Then you can merge the result of these two branches and possibly add a few other layers afterwards to act a the final classifier/regressor. To build such a model in Keras you need to use the functional API instead. Just for demonstration, here is an example:
inp_img = Input(shape=image_shape)
inp_others = Input(shape=others_shape)
# branch 1: process input image
x = Conv2D(...)(inp_img)
x = Conv2D(...)(x)
x = MaxPool2D(...)(x)
out_b1 = Flatten()(x)
# branch 2: process other input
out_b2 = Dense(...)(inp_other)
# merge the results by concatenation
merged = concatenate([out_b1, out_b2])
# pass merged tensor to some other layers
x = Dense(...)(merged)
output = Dense(...)(x)
# build the model and compile it
model = Model([inp_img, inp_other], output)
model.compile(...)
# fit on training data
model.fit([img_array, other_array], label_array, ...)
Note that we used concatenation
layer above, but there are other merge layers which you can use. And make sure you read the functional API guide, it's a must-read guide.
add a comment |
One approach is to use a two branch model, where one branch processes the image and another branch processes other non-image inputs (such as posts texts or number of followers and followings, etc.). Then you can merge the result of these two branches and possibly add a few other layers afterwards to act a the final classifier/regressor. To build such a model in Keras you need to use the functional API instead. Just for demonstration, here is an example:
inp_img = Input(shape=image_shape)
inp_others = Input(shape=others_shape)
# branch 1: process input image
x = Conv2D(...)(inp_img)
x = Conv2D(...)(x)
x = MaxPool2D(...)(x)
out_b1 = Flatten()(x)
# branch 2: process other input
out_b2 = Dense(...)(inp_other)
# merge the results by concatenation
merged = concatenate([out_b1, out_b2])
# pass merged tensor to some other layers
x = Dense(...)(merged)
output = Dense(...)(x)
# build the model and compile it
model = Model([inp_img, inp_other], output)
model.compile(...)
# fit on training data
model.fit([img_array, other_array], label_array, ...)
Note that we used concatenation
layer above, but there are other merge layers which you can use. And make sure you read the functional API guide, it's a must-read guide.
One approach is to use a two branch model, where one branch processes the image and another branch processes other non-image inputs (such as posts texts or number of followers and followings, etc.). Then you can merge the result of these two branches and possibly add a few other layers afterwards to act a the final classifier/regressor. To build such a model in Keras you need to use the functional API instead. Just for demonstration, here is an example:
inp_img = Input(shape=image_shape)
inp_others = Input(shape=others_shape)
# branch 1: process input image
x = Conv2D(...)(inp_img)
x = Conv2D(...)(x)
x = MaxPool2D(...)(x)
out_b1 = Flatten()(x)
# branch 2: process other input
out_b2 = Dense(...)(inp_other)
# merge the results by concatenation
merged = concatenate([out_b1, out_b2])
# pass merged tensor to some other layers
x = Dense(...)(merged)
output = Dense(...)(x)
# build the model and compile it
model = Model([inp_img, inp_other], output)
model.compile(...)
# fit on training data
model.fit([img_array, other_array], label_array, ...)
Note that we used concatenation
layer above, but there are other merge layers which you can use. And make sure you read the functional API guide, it's a must-read guide.
answered Nov 21 '18 at 7:42
todaytoday
11.2k22239
11.2k22239
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400895%2fhow-to-add-a-second-input-argument-the-first-is-an-image-to-a-cnn-model-built%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown